Greenhouse Development Update 2 - May

That computer can browse the web, but "the web" (or, web users around the globe) typically can't browse files or web applications running on the computer. Publishing a server on the internet takes work, costs money, and requires maintenance and upkeep. It can be especially difficult if you want to use your computer, not rent someone else's. If you live in a dormitory or if you get internet access via wifi from your neighbor or via tethering to your phone, you may not be able to perform the necessary modifications to your router's configuration even if you knew how.

Greenhouse provides an easy way to make one or more computers into servers that anyone in the world can connect to securely & reliably, regardless of where or how the servers are connected to the internet.

Greenhouse is being designed from the ground up as a trustless service, that is, you don't have to trust me or whoever's running the service to keep your data secure — it's designed so it can't access your data in the first place.

For more information about why I am building this, see The "Pragmatic Path" 4-Year Update: Introducing Greenhouse!

You may also check out the source code at git.sequentialread.com/forest/greenhouse

Previous post in this series: Greenhouse development update 1 - April

Next post in this series: Greenhouse Update 3 - August

For the month of May I've recorded some screencasts and I'm going to try embedding them as HTML videos here. There is no audio included. If you cannot see the video, please leave a comment below.

1. first boot & admin registration

48 seconds

In this screencast, I'm acting as the greenhouse cloud service operator launching the service for the first time.

• 00:09 I start up the greenhouse web application for the first time
  • For now, the application is only avaliable on my computer (localhost)
• 00:29 I register an account
  • The first account registered will automatically be the administrator account.
• 00:42 I confirm my email, completing the registration process

Next up, here are some technical details about the next screencast.

I have created a new background service called greenhouse-daemon which is used by the desktop app. I also replaced the existing "greenhouse hosts itself" code with the daemon, now the greenhouse web application is powered by the same underlying process that we offer to greenhouse users. This is part of the software development practice known as "dogfooding" or "eat your own dog food". By using my own technology from the start, I ensure that at very least, my code is appliciable to my own use-case and I get a lot of built-in opportunities to perform my own testing and QA, discover features I wish I had and trip over bugs and design issues before my users do.

The greenhouse-daemon provides a simple API to manage threshold client and my custom build of caddy server. The user must submit a request with a valid greenhouse API token to register the daemon with the greenhouse cloud service & once registered, the user can submit requests to the daemon to publish any locally accessible listener (server) or publish a directory full of files to the internet.

The daemon talks to threshold, which creates network tunnels to enable connectivity for requests from anyone anywhere in the world. It also talks to caddy, which will take care of obtaining valid TLS certificates + terminating TLS properly to ensure that all greenhouse users can host secure services easily.

I even wrote a self-testing routine for threshold to validate that the tunnels are up and running, with any potential errors or failures reported in detail before the daemon is allowed to proceed.

Currently the greenhouse daemon supports HTTPS, TLS, and TCP protocols. It leverages threshold's haproxy "PROXY" protocol support to provide the greenhouse user's server with the IP address that the thier client dialed from. This is important because the greenhouse users' applications may wish to log or otherwise use the remote IP address information for security, debugging, or anti-spam reasons.

2. launch a public-cloud-based gateway server & publish greenhouse to the internet

1 minute 37 seconds

In this screencast, I'm acting as the greenhouse cloud service operator setting up the public-cloud part of the service and using it to publish the greenhouse web application to the internet.

• 00:05 use the administrator account from the previous video to access the admin panel
• 00:11 tell greenhouse to rebalance its public internet gateway servers
• 00:21 watching digital ocean create the VM for the first worker 😴
  • waiting for the VM's ssh host public key to be avaliable for download. This prevents SSH man-in-the-middle attacks
  • this part takes about 1-2 minutes, most of the waiting was cut.
• 00:32 watching greenhouse log into the new worker VM and run a script to install threshold in server mode 😴
• 01:00 scroll down to see the visual representation of the internet gateway server, including the list of tenants allocated to it (only tenant 1 for now) and the thermometers displaying current and predicted bandwidth usage for the month
• 01:04 tell greenhouse to boot up the greenhouse-daemon
  • check to make sure it started properly
• 01:16 tell greenhouse to configure the greenhouse-daemon
  • check to make sure the tunnel was created successfully
• 01:35 browse to the greenhouse web application via the new public URL https://greenhouse.forest-n-johnson.greenhouseusers.com
  • In the future I plan on hosting it under greenhouse.server.garden but I need to add more DNS features to the service before I can do that.
  • connection goes:

• • from my web browser to the public gateway server just created on digitalocean
  • into the threshold server process on the digitalocean VM, which routes it down the existing tunnel connection from the threshold client running on my laptop (threshold server uses the TLS ClientHello message's SNI (Server Name Indication) header to route to different clients/different tunnels)
  • into the threshold client process running on my laptop, which proxies the connection to caddy server on the laptop
  • into the caddy server process, which handles the encryption / security of the connection (TLS), and then proxies the connection to the local greenhouse server on the laptop. The caddy server also terminates the threshold server's PROXY protocol so it can create the X-Forwarded-For HTTP header
• Importantly, there's only one TLS session and it's between the web browser and the caddy server embedded in the greenhouse-daemon. This means that digitalocean themselves or anyone with access to the digitalocean VM cannot snoop on the contents of the TLS connection

Finally, I also recorded a screencast of what it might look like for a greenhouse user to publish a server or some local files using the desktop application. For the example server I used an nginx docker container, and for the example files I published an HTML line chart report documenting the growth of active user accounts and monthly revenue for cyberia's capsul cloud compute service.

3. as a user, I want my server to be online 😀

2 minutes 46 seconds

In this screencast, I am acting as a potential user of the greenhouse service who wants to publish a server on the internet.

• 00:03 I navigate to the greenhouse website
• 00:14 I log into my account
• 00:28 I create a new API token & copy it to the clipboard
• 00:42 I run the desktop application
  • For this demo it's run via the command line, but later it will be something the user can easily download from the greenhouse website and run/install from thier downloads folder.
• 00:48 I paste the API token into the desktop application to register it with my greenhouse account.
• 00:51 I create a new tunnel configuration and give it the test1 subdomain. I leave the default destination setting localhost:80
• 01:05 As an example for this demo, I use docker to run an instance of nginx HTTP server on my laptop, listening on port 80
  • Greenhouse can work with any server application. For example, if you are developing a React application with create-react-app (or any other local development server), you could use greenhouse to temporarily publish your local server on the internet
• 01:15 I connect to localhost in the web browser to demonstrate that the nginx server is running & avaliable locally
• 01:25 I apply the new tunnel configuration
• 01:39 I browse to https://test1.forest-n-johnson.greenhouseusers.com, demonstrating that my local nginx server is now published to the internet & secured by TLS. (It's using HTTPS, not HTTP)
• 01:45 I create a new tunnel configuration with test2 as the subdomain & switch it to the "serve local files" destination type.
• 01:57 I manually type in the path of the folder I want to publish: /home/forest/Desktop/capsul-stonks-dec2020
  • In the real application users will be able to open the file chooser and click on a folder instead.
  • They will also be sternly warned that everything inside this folder is about to become published on the internet.
• 02:19 I apply the new tunnel configuration
• 02:26 I browse to https://test2.forest-n-johnson.greenhouseusers.com, demonstrating that I can see a list of the files in the folder.
  • Note that I renamed index.html to index1.html for the demo. Normally caddy would serve index.html directly as the default document for the folder, however because it has been renamed, caddy will generate its own directory listing page. Later on this directory listing and other file server settings will be customizable.
• 02:38 I navigate to index1.html to display the chart.

Previous post in this series: Greenhouse development update 1 - April

Next post in this series: Greenhouse Update 3 - August

For more information about why I am building this, see The "Pragmatic Path" 4-Year Update: Introducing Greenhouse!

You may also check out the source code at git.sequentialread.com/forest/greenhouse