middleware and tlsOptions properties no longer exist? #29

New Issue

Ghost · 2020-08-03T05:52:16Z

Ghost commented

2020-08-03 05:52:16 +00:00

In the Automatic HTTPS example at https://github.com/solderjs/greenlock#automatic-https, it shows the returned greenlock instance to have middleware and tlsOptions properties, but when running, neither of these exist. I have also tried the greenlock-express package, but cannot find them there either.

Have they been removed? If so, is there an equivalent, or can you provide guidance on a simple implementation? Thanks.

In the Automatic HTTPS example at https://github.com/solderjs/greenlock#automatic-https, it shows the returned greenlock instance to have `middleware` and `tlsOptions` properties, but when running, neither of these exist. I have also tried the `greenlock-express` package, but cannot find them there either. Have they been removed? If so, is there an equivalent, or can you provide guidance on a simple implementation? Thanks.

coolaj86 commented

2020-08-03 06:35:44 +00:00

Take a look at the Quick Start here:
https://git.rootprojects.org/root/greenlock-express.js

And then take a look at the examples:
https://git.rootprojects.org/root/greenlock-express.js/src/branch/master/examples

In the prior version greenlock-express was just greenlock with a few default options set.

In this version the two are cleanly separated such that greenlock is only the ACME portion and greenlock-express is the HTTP portion.

What are you trying to do? What node HTTP framework are you using?

Take a look at the Quick Start here: https://git.rootprojects.org/root/greenlock-express.js And then take a look at the examples: https://git.rootprojects.org/root/greenlock-express.js/src/branch/master/examples In the prior version greenlock-express was just greenlock with a few default options set. In this version the two are cleanly separated such that greenlock is only the ACME portion and greenlock-express is the HTTP portion. What are you trying to do? What node HTTP framework are you using?

Ghost commented

2020-08-03 06:51:45 +00:00

I'm trying to integrate it into my own framework that operates primarily over a websocket. Originally, the bootstrapper would fire up a plain Node HTTPS server that hosts static content using serve-static from the build directory (e.g. react build output) and attach a websocket server to it. It would also fire up a simple HTTP server to redirect to HTTPS.

Ideally, I'd like to avoid introducing express as a dependency and just use serve-static. I'm pretty sure this is still possible with the implementation as it stands, just trying to nut it out. Any help would be greatly appreciated!

I'm trying to integrate it into my own framework that operates primarily over a websocket. Originally, the bootstrapper would fire up a plain Node HTTPS server that hosts static content using `serve-static` from the `build` directory (e.g. react build output) and attach a websocket server to it. It would also fire up a simple HTTP server to redirect to HTTPS. Ideally, I'd like to avoid introducing `express` as a dependency and just use `serve-static`. I'm pretty sure this is still possible with the implementation as it stands, just trying to nut it out. Any help would be greatly appreciated!

Ghost commented

2020-08-03 07:05:59 +00:00

I'm looking at the websockets example now, if all goes well, it should integrate OK just by passing the serve-static middleware instance to serveApp, though there are a few configuration options I need to figure out. I'll also be adding some additional endpoints to the HTTPS server, but adding them should be no different to adding them to a standard HTTPS server. I'll keep you posted.

I'm looking at the websockets example now, if all goes well, it should integrate OK just by passing the `serve-static` middleware instance to `serveApp`, though there are a few configuration options I need to figure out. I'll also be adding some additional endpoints to the HTTPS server, but adding them should be no different to adding them to a standard HTTPS server. I'll keep you posted.

coolaj86 commented

2020-08-03 07:29:08 +00:00

FYI: Greenlock has never had any dependency on express.js (other than a devDependency for one of the examples).

Sounds good.

I like to follow the pattern of putting a require.main listener in my app.js for when I need to test it without HTTPS:

if (require.main === module) {
  require('http').createServer(app).listen(process.env.PORT || 3000, function () {
    console.log("Listening on", this.address())
  });
}

FYI: Greenlock has never had any dependency on express.js (other than a `devDependency` for one of the examples). Sounds good. I like to follow the pattern of putting a `require.main` listener in my `app.js` for when I need to test it without HTTPS: ```js if (require.main === module) { require('http').createServer(app).listen(process.env.PORT || 3000, function () { console.log("Listening on", this.address()) }); } ```

Ghost commented

2020-08-03 08:32:29 +00:00

OK, next issue... Using the CLI is not an option, I need to add domains to the manager programmatically, but I can't find how to get access to the manager instance to call add.

It doesn't seem to be exposed as a result of the call to init, nor in the callback passed to ready. There don't seem to be any greenlock-express examples that handle adding domains programmatically. I've tried creating the manager manually, but am struggling to create an instance and pass it in.

Can you point me in the right direction? I'm struggling here a bit.

OK, next issue... Using the CLI is not an option, I need to add domains to the manager programmatically, but I can't find how to get access to the manager instance to call `add`. It doesn't seem to be exposed as a result of the call to `init`, nor in the callback passed to `ready`. There don't seem to be any `greenlock-express` examples that handle adding domains programmatically. I've tried creating the manager manually, but am struggling to create an instance and pass it in. Can you point me in the right direction? I'm struggling here a bit.

coolaj86 commented

2020-08-03 08:43:06 +00:00

greenlock and greenlock-express are completely separate.

See https://git.rootprojects.org/root/greenlock.js#user-content-quick-start

Just make sure that anywhere you initialize Greenlock Express or Greenlock that the packageRoot and configDir values agree between instances, otherwise you'll get different configurations.

`greenlock` and `greenlock-express` are completely separate. See https://git.rootprojects.org/root/greenlock.js#user-content-quick-start Just make sure that anywhere you initialize Greenlock Express or Greenlock that the `packageRoot` and `configDir` values agree between instances, otherwise you'll get different configurations.

Ghost commented

2020-08-03 08:46:17 +00:00

I have to be honest here... I'm almost ready to give up and go back to using certbot and nginx. It's clear the documentation is not up to scratch, most of it is out of date, and it shows from the issues that have been raised. I'm struggling to thread my way through debugging sessions and currently can't see a way forward.

I've spent all day on this now, not a cost I had anticipated!

If you can help me get up and running, I will try to find the time to submit a PR or two to help with the state of the documentation, but at the moment, I'm getting frustrated.

I have to be honest here... I'm almost ready to give up and go back to using certbot and nginx. It's clear the documentation is not up to scratch, most of it is out of date, and it shows from the issues that have been raised. I'm struggling to thread my way through debugging sessions and currently can't see a way forward. I've spent all day on this now, not a cost I had anticipated! If you can help me get up and running, I will try to find the time to submit a PR or two to help with the state of the documentation, but at the moment, I'm getting frustrated.

Ghost commented

2020-08-03 08:47:08 +00:00

OK, so if I create an instance of greenlock manually, how do I then pass it to greenlock-express?

OK, so if I create an instance of `greenlock` manually, how do I then pass it to `greenlock-express`?

coolaj86 commented

2020-08-03 08:49:39 +00:00

Actually, it sounds like what you want is just to override the manager callbacks, in which case you should look at:

https://git.rootprojects.org/root/greenlock-manager-test.js

You only need to implement enough to pass the tests for the features you need.

For example, if you don't need wildcard support, your callback functions don't need to pass those tests.

Public vs Private

Public

If you're going to publish your module to NPM, just pass those tests and do so. The config is JSON (so that it can be stored on the file system, or in a database), which is why you specify the name of the module, rather than require it.

Private

Part of the reason for packageRoot is that npm doesn't handle in-project dependencies very well (or at all), and since many people want implementations that can't be published as public modules, the packageRoot is the source from which paths will be required.

It feels a little weird, but this is also what haraka and many other projects that have to handle unpublished or private dependencies have to do.

Actually, it sounds like what you want is just to override the manager callbacks, in which case you should look at: https://git.rootprojects.org/root/greenlock-manager-test.js You only need to implement enough to pass the tests for the features you need. For example, if you don't need wildcard support, your callback functions don't need to pass those tests. ### Public vs Private **Public** If you're going to publish your module to NPM, just pass those tests and do so. The config is JSON (so that it can be stored on the file system, or in a database), which is why you specify the name of the module, rather than `require` it. **Private** Part of the reason for `packageRoot` is that `npm` doesn't handle in-project dependencies very well (or at all), and since many people want implementations that can't be published as public modules, the `packageRoot` is the source from which paths will be `require`d. It feels a little weird, but this is also what `haraka` and many other projects that have to handle unpublished or private dependencies have to do.

coolaj86 commented

2020-08-03 08:50:15 +00:00

OK, so if I create an instance of greenlock manually, how do I then pass it to greenlock-express?

You don't. You specify the same packageRoot and configDir.

> OK, so if I create an instance of greenlock manually, how do I then pass it to greenlock-express? You don't. You specify the same `packageRoot` and `configDir`.

Ghost commented

2020-08-03 08:55:38 +00:00

I'm even more confused. Do I really need to implement my own manager to add domains programmatically? Can't I just get at the either the greenlock instance and call greenlock.sites.add, or get at the manager and call manager.add?

All I want to do is use the middleware from greenlock-express and programmatically add a domain - I cannot use the CLI to do this, it is not an option.

Can you provide a code sample of how to do this?

I'm even more confused. Do I really need to implement my own manager to add domains programmatically? Can't I just get at the either the greenlock instance and call `greenlock.sites.add`, or get at the manager and call `manager.add`? All I want to do is use the middleware from `greenlock-express` and programmatically add a domain - I *cannot* use the CLI to do this, it is not an option. Can you provide a code sample of how to do this?

coolaj86 commented

2020-08-03 08:58:50 +00:00

Just follow the Quick Start that I linked you to:

'use strict';

var pkg = require('./package.json');
var Greenlock = require('greenlock');
var greenlock = Greenlock.create({
    packageRoot: __dirname,
    configDir: "./greenlock.d/",
    packageAgent: pkg.name + '/' + pkg.version,
    maintainerEmail: pkg.author,
    staging: true,
    notify: function(event, details) {
        if ('error' === event) {
            // `details` is an error object in this case
            console.error(details);
        }
    }
});

var altnames = ['example.com', 'www.example.com'];

greenlock
    .add({
        subject: altnames[0],
        altnames: altnames
    })
    .then(function() {
        // saved config to db (or file system)
    });

Just follow the Quick Start that I linked you to: ```js 'use strict'; var pkg = require('./package.json'); var Greenlock = require('greenlock'); var greenlock = Greenlock.create({ packageRoot: __dirname, configDir: "./greenlock.d/", packageAgent: pkg.name + '/' + pkg.version, maintainerEmail: pkg.author, staging: true, notify: function(event, details) { if ('error' === event) { // `details` is an error object in this case console.error(details); } } }); var altnames = ['example.com', 'www.example.com']; greenlock .add({ subject: altnames[0], altnames: altnames }) .then(function() { // saved config to db (or file system) }); ```

Ghost commented

2020-08-03 08:59:43 +00:00

Thanks. How do I then use this greenlock instance with greenlock-express???

Thanks. How do I then use this greenlock instance with `greenlock-express`???

coolaj86 commented

2020-08-03 09:02:56 +00:00

If you're using your own database and need custom functionality, you write the 2 to 4 callback functions that you need (i.e. for set, get, remove, list).

If you just want to use the default file-based storage, then you don't need to do that.

If you're building out a hosting service (which is the reason most people need dynamic certificates), then you probably have your stuff in a database (or will soon).

Just start with whichever way is simplest for you and don't do any special customization or configuration until you need to.

If you're using your own database and need custom functionality, you write the 2 to 4 callback functions that you need (i.e. for set, get, remove, list). If you just want to use the default file-based storage, then you don't need to do that. ---- If you're building out a hosting service (which is the reason most people need dynamic certificates), then you probably have your stuff in a database (or will soon). Just start with whichever way is simplest for you and don't do any special customization or configuration until you need to.

coolaj86 commented

2020-08-03 09:03:59 +00:00

Thanks. How do I then use this greenlock instance with greenlock-express???

You just set configDir and packageRoot to the same thing.

That's it.

You don't pass one to the other.

> Thanks. How do I then use this greenlock instance with greenlock-express??? You just set `configDir` and `packageRoot` to the same thing. That's it. You don't pass one to the other.

Ghost commented

2020-08-03 09:05:57 +00:00

Argh. I am not explaining myself. I don't want custom database or any custom functionality.

I simply want to use the middleware from greenlock-express and programmatically add a domain.

You have shown me how to create an instance of greenlock and programatically add a domain, but NOT how to then use this instance of greenlock with the express middleware.

Please... A simple example of what I am asking would be great.

Argh. I am not explaining myself. I don't want custom database or any custom functionality. *I simply want to use the middleware from `greenlock-express` and programmatically add a domain.* You have shown me how to create an instance of `greenlock` and programatically add a domain, but *NOT* how to then use this instance of greenlock with the express middleware. Please... A simple example of what I am asking would be great.

coolaj86 commented

2020-08-03 09:06:30 +00:00

Let's say you have a route /add-a-site where you do some admin authentication and add a site, you'd do it like this:

app.js:

var greenlock = ... /* shown just above */

app.use('/add-a-site', function (req, res) {
  greenlock.add({ ... }).then(function () {
    res.end("Happy day!");
  });
});

It writes to the file system (or database) and then when Greenlock Express goes to fetch a certificate, it'll find that it's been added and prepare the tlsOptions accordingly.

Let's say you have a route `/add-a-site` where you do some admin authentication and add a site, you'd do it like this: `app.js`: ``` var greenlock = ... /* shown just above */ app.use('/add-a-site', function (req, res) { greenlock.add({ ... }).then(function () { res.end("Happy day!"); }); }); ``` It writes to the file system (or database) and then when Greenlock Express goes to fetch a certificate, it'll find that it's been added and prepare the `tlsOptions` accordingly.

Ghost commented

2020-08-03 09:07:28 +00:00

No routes, nothing like that, FFS... I am not using express...

coolaj86 commented

2020-08-03 09:11:21 +00:00

how to then use this instance of greenlock with the express middleware.

You don't.

They're separate.

If you want to use Greenlock Express and load your own express or koa middleware, you do that with https://git.rootprojects.org/root/greenlock-express.js#user-content-v4-quickstart

If you want to create admin middleware to manage with the bare Greenlock API, you do that with https://git.rootprojects.org/root/greenlock.js/src/branch/master/README.md#user-content-quick-start

You can have 600 instances of Greenlock and they'll all work just fine together as long as they have the same packageRoot and configDir (or they can even be in completely separate processes as long as the configDir has the same settings).

> how to then use this instance of greenlock with the express middleware. You don't. They're separate. --- If you want to use Greenlock Express and load your own express or koa middleware, you do that with https://git.rootprojects.org/root/greenlock-express.js#user-content-v4-quickstart If you want to create admin middleware to manage with the bare Greenlock API, you do that with https://git.rootprojects.org/root/greenlock.js/src/branch/master/README.md#user-content-quick-start You can have 600 instances of Greenlock and they'll all work just fine together as long as they have the same `packageRoot` and `configDir` (or they can even be in completely separate processes as long as the `configDir` has the same settings).

Ghost commented

2020-08-03 09:11:36 +00:00

sigh OK, I think I get what you are saying - If I create an instance of greenlock and call sites.add, wait for the promise to complete, and only then simply use greenlock-express, and it will pick the site up from the saved configuration file.

Assuming this is correct, sorry it took so long for me to grasp this, but this is not an intuitive way of programming to me.

I will keep you posted on progress.

*sigh* OK, I think I get what you are saying - If I create an instance of greenlock and call sites.add, wait for the promise to complete, and only then simply use greenlock-express, and it will pick the site up from the saved configuration file. Assuming this is correct, sorry it took so long for me to grasp this, but this is not an intuitive way of programming to me. I will keep you posted on progress.

coolaj86 commented

2020-08-03 09:12:20 +00:00

No routes, nothing like that, FFS... I am not using express...

Then what are you using?

If you just use node's basic HTTP, it's about the same:

app.js:

var greenlock = ... /* shown just above */

module.exports = function (req, res) {
  greenlock.add({ ... }).then(function () {
    res.end("Happy day!");
  });
};

> No routes, nothing like that, FFS... I am not using express... Then what are you using? If you just use node's basic HTTP, it's about the same: `app.js:` ``` var greenlock = ... /* shown just above */ module.exports = function (req, res) { greenlock.add({ ... }).then(function () { res.end("Happy day!"); }); }; ```

coolaj86 commented

2020-08-03 09:24:20 +00:00

sorry it took so long for me to grasp this

And I'm sorry it took me so long to explain it.

but this is not an intuitive way of programming to me.

That's the problem of creating something that scales beyond a single process.

If you only care about something working on exactly one machine on exactly one CPU, and you don't plan to grow it as a service with many users, then you can just put everything in JavaScript and call it a day.

If, however, you're running a service that will gain more users and therefore eventually have to run in 2+ separate processes, you have to think about things differently - you have to put "data" in serializable formats, like JSON, so that more than one process can access it at a time (and preferrably in a database, so that more than one machine can access it at a time).

I feel like Greenlock Express is very very good for the simplest cases (run a blog, or personal business site), and Greenlock is very very good at complex cases (like a hosting service with 1,000 users), but that middle ground of a hosting service with 10-100 users... it's a weird place - because it's basically as complicated as 1,000 user case, but seems like it shouldn't be... but it actually is, it's just not obvious that it is.

The difference between 1 and 2 is much bigger than the difference between 2 and 10,000.

> sorry it took so long for me to grasp this And I'm sorry it took me so long to explain it. > but this is not an intuitive way of programming to me. That's the problem of creating something that scales beyond a single process. If you only care about something working on exactly one machine on exactly one CPU, and you don't plan to grow it as a service with many users, then you can just put everything in JavaScript and call it a day. If, however, you're running a service that will gain more users and therefore eventually have to run in 2+ separate processes, you have to think about things differently - you have to put "data" in serializable formats, like JSON, so that more than one process can access it at a time (and preferrably in a database, so that more than one machine can access it at a time). ---- I feel like Greenlock Express is very very good for the simplest cases (run a blog, or personal business site), and Greenlock is very very good at complex cases (like a hosting service with 1,000 users), but that middle ground of a hosting service with 10-100 users... it's a weird place - because it's basically as complicated as 1,000 user case, but seems like it _shouldn't_ be... but it actually is, it's just not _obvious_ that it is. The difference between 1 and 2 is much bigger than the difference between 2 and 10,000.

Ghost commented

2020-08-03 09:35:57 +00:00

I totally understand that Node.js is single threaded, and the need for storing config externally so that other processes can access it. This is not ASP.NET haha. Give me a little while to get it all working and I'll elaborate a bit on my perspective.

I really appreciate the immediate response and the time you have spent with me.

I totally understand that Node.js is single threaded, and the need for storing config externally so that other processes can access it. This is not ASP.NET haha. Give me a little while to get it all working and I'll elaborate a bit on my perspective. I really appreciate the immediate response and the time you have spent with me.

coolaj86 commented

2020-08-03 09:41:44 +00:00

I really appreciate the immediate response and the time you have spent with me.

You're welcome, and lucky. :D

You happened to catch me at a good time.

I want to be better about responding quickly in general, but usually I'm in the middle of something else that needs my full attention and by the end of the 4-hour block of coding I've forgotten or the email is buried under 20 others. :-/

> I really appreciate the immediate response and the time you have spent with me. You're welcome, and lucky. :D You happened to catch me at a good time. I want to be better about responding quickly in general, but usually I'm in the middle of something else that needs my full attention and by the end of the 4-hour block of coding I've forgotten or the email is buried under 20 others. :-/

Sign in to join this conversation.

No Label

No Milestone

No Assignees

2 Participants

Notifications

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: root/greenlock.js#29

middleware and tlsOptions properties no longer exist? #29

Public vs Private

`middleware` and `tlsOptions` properties no longer exist? #29