`middleware` and `tlsOptions` properties no longer exist? #29

In the Automatic HTTPS example at https://github.com/solderjs/greenlock#automatic-https, it shows the returned greenlock instance to have middleware and tlsOptions properties, but when running, neither of these exist. I have also tried the greenlock-express package, but cannot find them there either.

Have they been removed? If so, is there an equivalent, or can you provide guidance on a simple implementation? Thanks.

Take a look at the Quick Start here:
https://git.rootprojects.org/root/greenlock-express.js

And then take a look at the examples:
https://git.rootprojects.org/root/greenlock-express.js/src/branch/master/examples

In the prior version greenlock-express was just greenlock with a few default options set.

In this version the two are cleanly separated such that greenlock is only the ACME portion and greenlock-express is the HTTP portion.

What are you trying to do? What node HTTP framework are you using?

I'm trying to integrate it into my own framework that operates primarily over a websocket. Originally, the bootstrapper would fire up a plain Node HTTPS server that hosts static content using serve-static from the build directory (e.g. react build output) and attach a websocket server to it. It would also fire up a simple HTTP server to redirect to HTTPS.

Ideally, I'd like to avoid introducing express as a dependency and just use serve-static. I'm pretty sure this is still possible with the implementation as it stands, just trying to nut it out. Any help would be greatly appreciated!

I'm looking at the websockets example now, if all goes well, it should integrate OK just by passing the serve-static middleware instance to serveApp, though there are a few configuration options I need to figure out. I'll also be adding some additional endpoints to the HTTPS server, but adding them should be no different to adding them to a standard HTTPS server. I'll keep you posted.

FYI: Greenlock has never had any dependency on express.js (other than a devDependency for one of the examples).

Sounds good.

I like to follow the pattern of putting a require.main listener in my app.js for when I need to test it without HTTPS:

if (require.main === module) {
  require('http').createServer(app).listen(process.env.PORT || 3000, function () {
    console.log("Listening on", this.address())
  });
}

OK, next issue... Using the CLI is not an option, I need to add domains to the manager programmatically, but I can't find how to get access to the manager instance to call add.

It doesn't seem to be exposed as a result of the call to init, nor in the callback passed to ready. There don't seem to be any greenlock-express examples that handle adding domains programmatically. I've tried creating the manager manually, but am struggling to create an instance and pass it in.

Can you point me in the right direction? I'm struggling here a bit.

greenlock and greenlock-express are completely separate.

See https://git.rootprojects.org/root/greenlock.js#user-content-quick-start

Just make sure that anywhere you initialize Greenlock Express or Greenlock that the packageRoot and configDir values agree between instances, otherwise you'll get different configurations.

I have to be honest here... I'm almost ready to give up and go back to using certbot and nginx. It's clear the documentation is not up to scratch, most of it is out of date, and it shows from the issues that have been raised. I'm struggling to thread my way through debugging sessions and currently can't see a way forward.

I've spent all day on this now, not a cost I had anticipated!

If you can help me get up and running, I will try to find the time to submit a PR or two to help with the state of the documentation, but at the moment, I'm getting frustrated.

OK, so if I create an instance of greenlock manually, how do I then pass it to greenlock-express?

Actually, it sounds like what you want is just to override the manager callbacks, in which case you should look at:

https://git.rootprojects.org/root/greenlock-manager-test.js

You only need to implement enough to pass the tests for the features you need.

For example, if you don't need wildcard support, your callback functions don't need to pass those tests.

Public vs Private

Public

If you're going to publish your module to NPM, just pass those tests and do so. The config is JSON (so that it can be stored on the file system, or in a database), which is why you specify the name of the module, rather than require it.

Private

Part of the reason for packageRoot is that npm doesn't handle in-project dependencies very well (or at all), and since many people want implementations that can't be published as public modules, the packageRoot is the source from which paths will be required.

It feels a little weird, but this is also what haraka and many other projects that have to handle unpublished or private dependencies have to do.

OK, so if I create an instance of greenlock manually, how do I then pass it to greenlock-express?

You don't. You specify the same packageRoot and configDir.

I'm even more confused. Do I really need to implement my own manager to add domains programmatically? Can't I just get at the either the greenlock instance and call greenlock.sites.add, or get at the manager and call manager.add?

All I want to do is use the middleware from greenlock-express and programmatically add a domain - I cannot use the CLI to do this, it is not an option.

Can you provide a code sample of how to do this?

Just follow the Quick Start that I linked you to:

'use strict';

var pkg = require('./package.json');
var Greenlock = require('greenlock');
var greenlock = Greenlock.create({
    packageRoot: __dirname,
    configDir: "./greenlock.d/",
    packageAgent: pkg.name + '/' + pkg.version,
    maintainerEmail: pkg.author,
    staging: true,
    notify: function(event, details) {
        if ('error' === event) {
            // `details` is an error object in this case
            console.error(details);
        }
    }
});

var altnames = ['example.com', 'www.example.com'];

greenlock
    .add({
        subject: altnames[0],
        altnames: altnames
    })
    .then(function() {
        // saved config to db (or file system)
    });

Thanks. How do I then use this greenlock instance with greenlock-express???

If you're using your own database and need custom functionality, you write the 2 to 4 callback functions that you need (i.e. for set, get, remove, list).

If you just want to use the default file-based storage, then you don't need to do that.

If you're building out a hosting service (which is the reason most people need dynamic certificates), then you probably have your stuff in a database (or will soon).

Just start with whichever way is simplest for you and don't do any special customization or configuration until you need to.

Thanks. How do I then use this greenlock instance with greenlock-express???

You just set configDir and packageRoot to the same thing.

That's it.

You don't pass one to the other.

Argh. I am not explaining myself. I don't want custom database or any custom functionality.

I simply want to use the middleware from greenlock-express and programmatically add a domain.

You have shown me how to create an instance of greenlock and programatically add a domain, but NOT how to then use this instance of greenlock with the express middleware.

Please... A simple example of what I am asking would be great.

Let's say you have a route /add-a-site where you do some admin authentication and add a site, you'd do it like this:

app.js:

var greenlock = ... /* shown just above */

app.use('/add-a-site', function (req, res) {
  greenlock.add({ ... }).then(function () {
    res.end("Happy day!");
  });
});

It writes to the file system (or database) and then when Greenlock Express goes to fetch a certificate, it'll find that it's been added and prepare the tlsOptions accordingly.

No routes, nothing like that, FFS... I am not using express...

how to then use this instance of greenlock with the express middleware.

You don't.

They're separate.

If you want to use Greenlock Express and load your own express or koa middleware, you do that with https://git.rootprojects.org/root/greenlock-express.js#user-content-v4-quickstart

If you want to create admin middleware to manage with the bare Greenlock API, you do that with https://git.rootprojects.org/root/greenlock.js/src/branch/master/README.md#user-content-quick-start

You can have 600 instances of Greenlock and they'll all work just fine together as long as they have the same packageRoot and configDir (or they can even be in completely separate processes as long as the configDir has the same settings).

sigh OK, I think I get what you are saying - If I create an instance of greenlock and call sites.add, wait for the promise to complete, and only then simply use greenlock-express, and it will pick the site up from the saved configuration file.

Assuming this is correct, sorry it took so long for me to grasp this, but this is not an intuitive way of programming to me.

I will keep you posted on progress.

No routes, nothing like that, FFS... I am not using express...

Then what are you using?

If you just use node's basic HTTP, it's about the same:

app.js:

var greenlock = ... /* shown just above */

module.exports = function (req, res) {
  greenlock.add({ ... }).then(function () {
    res.end("Happy day!");
  });
};

sorry it took so long for me to grasp this

And I'm sorry it took me so long to explain it.

but this is not an intuitive way of programming to me.

That's the problem of creating something that scales beyond a single process.

If you only care about something working on exactly one machine on exactly one CPU, and you don't plan to grow it as a service with many users, then you can just put everything in JavaScript and call it a day.

If, however, you're running a service that will gain more users and therefore eventually have to run in 2+ separate processes, you have to think about things differently - you have to put "data" in serializable formats, like JSON, so that more than one process can access it at a time (and preferrably in a database, so that more than one machine can access it at a time).

I feel like Greenlock Express is very very good for the simplest cases (run a blog, or personal business site), and Greenlock is very very good at complex cases (like a hosting service with 1,000 users), but that middle ground of a hosting service with 10-100 users... it's a weird place - because it's basically as complicated as 1,000 user case, but seems like it shouldn't be... but it actually is, it's just not obvious that it is.

The difference between 1 and 2 is much bigger than the difference between 2 and 10,000.

I totally understand that Node.js is single threaded, and the need for storing config externally so that other processes can access it. This is not ASP.NET haha. Give me a little while to get it all working and I'll elaborate a bit on my perspective.

I really appreciate the immediate response and the time you have spent with me.

I really appreciate the immediate response and the time you have spent with me.

You're welcome, and lucky. :D

You happened to catch me at a good time.

I want to be better about responding quickly in general, but usually I'm in the middle of something else that needs my full attention and by the end of the 4-hour block of coding I've forgotten or the email is buried under 20 others. :-/