Untitled RSS Feed

Getting started with a Serverless Github App

Wed, 05 May 2021 14:00:00 GMT

If you've considered enhancing your teams workflow with a Github app, or even developing something for a wider audience, the Github API is very good. Nevertheless, it can be a little tricky if you want to use serverless-node as the official tutorial is in Ruby and the Official Octokit node packages is catered to a more server-full solution (i.e. they have middleware for handling auth, which is not super applicable to serverless functions).

We're going to walk through how to set up a very basic Github app that's able to comment on Pull requests in response to a web hook, so that we have an end-to-end example from code to comment and you can take it from there. Finished repo is here.

Code Setup

We're going to use Redwood as an example, mostly because it makes getting setup and coding an absolute breeze, but you should have no trouble following along with this tutorial if you have a prefered node base serverless setup (netlify, serverless etc).

'cd' into a new folder where you want your app to go and run 'yarn create redwood-app ./'. This will generate a lot of code as it's setting up a fullstack app, but we're only concerned about 'api/src/functions/', creating a new file called 'githubhook.js'.

export const handler = () => {
  console.log('hi')
  return {
    statusCode: 200,
    body: 'hello'
  }
}

Spin it up with 'yarn rw dev'. It's going to serve the redwood frontend on port 8910, but the functions will be on 8911 so go to localhost:8911/githubhook and you should see 'hello' in the browser and 'hi' in the terminal.

Great, our function is setup. The reason we've set up a function is Github apps work by listening to web hooks that Github sends to our application (or function in this case). In response to these webhooks we can do most other actions on Github (leave comments, close issues etc). Before we turn out attention to Github, let's make a quick change to our handler so that we can see info about incoming web hooks. Change it to:

export const handler = (req) => {
  const theirSignature = req.headers['x-hub-signature-256']
  const eventType = req.headers['x-github-event']
  const event = JSON.parse(req.body)
  console.log(`
theirSignature is ${theirSignature}
eventType is ${eventType}
installation id is ${event.installation.id}
  `)
  return {
    statusCode: 200,
  }
}

The event type should show us if it's an 'installation' event or 'pull_request' event. Now let's set up Github.

Creating a Github app

For longevity, I'll leave a link to the official tutorial. Let's get the app settings ready in 11 quick steps

1. Go to smee.io and click "start new channel" to get yourself a unique url, we'll use this to forward webhooks to your local app, then install and run the smee client in a new terminal.

npm install --global smee-client
smee --url https://smee.io/<your-unique-hash> --path /githubhook --port 8911

2. Go to app settings and click "New Github App".

3. Give your app a name.

4. Put your smee url into the home page URL (you can change this later, but for the time being you need to put something in this field).

5. Put your smee url into the webhook URL and add a secret (the secret is optional but we're going to cover it). I normally use a password generater to create a secret.

6. Create a '.env' file in the root of your project. You should see '.env.example' already generated for you. Once you create '.env' you should notice that it's already gitignored. This is good as we don't want to commit our secrets. Add a new variable for our secret like so.

GITHUB_APP_SECRET=d7VtfAC2Uk3G49K4hygTUktzNkvyKx

7. Under repositry settings, give your app "read" permission for metadata, and "read and write" permission for pull requests.

8. Subscribe to pull-request events.

9. Click the big green "Create Github App" button.

10. Add your app id to the '.env' file:

GITHUB_APP_SECRET=d7VtfAC2Uk3G49K4hygTUktzNkvyKx

GITHUB_APP_ID=113860

11. Generate a private key.

You will then download a '.pem' file. Open it in a text editor as we need to paste the contents into our '.env' file. Unfortunatly with Node we can't use multiline environment variables, so you will have to replace all of the new lines with escaped new lines instead (i.e. '\n'). Your .env file should look something like this now:

GITHUB_APP_SECRET=d7VtfAC2Uk3G49K4hygTUktzNkvyKx

GITHUB_APP_ID=113860

GITHUB_APP_PRIVATE_KEY="-----BEGIN RSA PRIVATE KEY-----\nMIIEowIBAAKCAQAA4+M/9x8bCQDI8h4CfRyWqXCyyrBIlNENhTDci51cq3JzRgoz\nNoj1A1ytvzO7pDnd3kkeQ2Vy9Yh/gB6dpsBc6r3w+40/apafzQh6RrpbSR6WCeOa\nRz8EmdbbnWwCmfDxB0YyLXvv9LIaYanNhlifbGnS5LMB7hPVsbiefKbF/UApbKnT\nME70qEhUpgtMsgFcDutH8Ru7h4sgMxUyK6trYj3IffNhOI6NCfKMzNh1zKENuKUr\nuMJvl0hdA4UpOwVzFe9N7xXsTHwGRZLKELeBS/UOoQH9fatTevSoZ7oNlsVbTd0q\nQ6G/5rkrFp3ThYMXrqi2KExn4TKKPaslqyoFIQIDAQABAoIBACjSoLbIH7OxLE4T\nCicXY/XedmjQw2/FM0LUye1Itz0PN48obJwsVJfRll5WChsVMqWLO5yfI8TQOubc\nlSk19G8or26gkuepK98y0ZSx9YBXtlD3ML/qjgxw7M56dszU2JiQ/pQfS5DuBsPQ\nAz05yvVEz76gQow/iVxY0itKRNVkvn2Vrh2cGRMwHtidLi6PVDe1P9PP1pkpzSGB\nO+k9qlAOTuOOP9h8GhDtT4IUN8uvoNkuXaX9YHYVoQfTKE51y5miAoMaDlAYJcJy\nJw6Loi9QP240UqAmNUHXZ+6Z0WC0M8D67gGwuHAfQRszWkEG7YlQFVTFx2syXvMb\n63MabAECgYEA8h9N8k5mn4el7nJn3umy+G6Te6GCZDH+PaEf3gUMybQz7ay4U5Q6\nnKG1dJvxxx5SF4HlwPSBW/FUg19RqqwmWWNL3yil6LIwmwAxpcqCDdAZSEc398oU\nrEHuMq2cuWLnYJOKzF/4q/es4NoRgRTFnxMmN/h1Ih+lehCTfELFuPECgYEA8PMU\nRlamrqJPJNsE4iTRKdGBPT3mZNT5g9xlNIWolbFW5VSmsyHBheKgWFA4XpyzsWCL\ndSNvmbjVml93FubVdf4NJN91Tchhy4JDXabhcHQ3j5u8ORDP10gZspClORvCUdaG\nBmQUqjNwC+fYLKobcRncXMMBe/zS4dQP7ikzjzECgYBeD3NEktijWRtJSwC3RKrW\ngH6jJNd2/UT7xECRC/0vzuXti5AASDGM7/WCW6LN7CWQJFKRZ2tpwJNIhhs/5qjv\nSPgMtcneYHspfCXNdqKXoyRvQ9umU8c8NFDJN1EPZDDm/+qIAzCj/hAXOiBauSsc\n5V+PluJKY2jxxsbFG1ucwQKBgQCQ/AOoK33SuVHcQHoYxcSiYDFfM38OD2Uwpg6z\n4vVFVdeO2TgRs+8p6+tGGMdCjxJFWm2wB6mgmyrU4DrdqfqqLDumg1uneTr3ZSO6\nF6+xpgzEuhYxVF9sEDN+UjFJQt3Ttr0g3Vnd7GOwlkpq3dTzYndJzgF3pPMT9jG7\nwkkHEQKBgD1+EqeubKxzmBvVinBssWKk7m26pg2Uf+tqe1Ox+CZhUi0aOfFL3NlT\nKpgTSA0iNtp9srSZrUtEWNZ/2ITLIZypBC1cMqbUG4r+VcWfLPKPrzLUuxWgJKr3\nggQlB6rrePyMxDL80uAcf6omuh1/s049XfdApQRs96SVw2kWLghw\n-----END RSA PRIVATE KEY-----"

Triggering a web hook

Okay we're ready to start building out the app. But first as a sanity check let's make sure we can recieve web hooks. It makes sense to start with an installation event since we need to install the app. Go to your app settings and click edit on your app. From there click "install app" on the left hand side and then the green "install" on the right hand side.

You will need to choose permissions for the app. Since it's your own app "all repositories" will be fine.

And with that we would have triggered a webhook. Check back in your terminal and you should see something like this:

api | eventType is installation
api | signature is sha256=f59dba407b01bbc18b625b8a27fe6d217c1602bf99bb0fc72c0838c20895f1ff
api | installation id is 16726859

If you plan on running the Github app in tandem with your own app, then you can store this installation id in your database, but for now we'll leave it. Let's now trigger a pull-request event. Make a small change to one of your non-critical repos (or create a test repo for this purpose). Editing a read me is an easy way to whip up a pull request.

And just like that we should have triggered another event and web hook. The terminal should contain something along the lines of this:

api | eventType is pull_request
api | signature is sha256=32be921ea95bd573802ceea398e0d868e6b0c04e6f848ff562f0c7addcfac481
api | installation id is 16726859

The key bit here is that the event is now 'pull_request'.

Verifying incoming web hooks

Let's handle security now. Because a web hook is nothing more than an endpoint exposed to the web waiting for another service to use it, we want to make sure requests come from Github before we proceed with taking any actions. We can do that using the signature. The signature is a SHA256 that's made with the secret and the body of the request, which means the correct signature can only be made if you know the secret. We can replicate the signature and verify it's Github sending the request. Let's put that in a function:

import { createHmac } from 'crypto'

const signRequestBody = (secret, body) =>
  'sha256=' + createHmac('sha256', secret)
  .update(body, 'utf-8')
  .digest('hex')

The 'crypto' module doesn't need to be installed as it's a core Node module. Now all we need to do is compare the signatures. Let's update our handler:

export const handler = async (req) => {
  const theirSignature = req.headers['x-hub-signature-256']
  const ourSignature = signRequestBody(process.env.GITHUB_APP_SECRET, req.body)
  if (theirSignature !== ourSignature) {
    return {
      statusCode: 401,
      body: 'Bad signature'
    }
  }
  const eventType = req.headers['x-github-event']
  const event = JSON.parse(req && req.body  || '{"installation": {}}')
  return {
    statusCode: 200,
  }
}

Now if you visit localhost:8911/githubhook in a browser, you should see "bad signature" as the browser is not signing its requests. Great!

Note, that if you're using Redwood, you can use their webhook helpers to verify the signature instead like so

import { verifyEvent } from '@redwoodjs/api/webhooks'

export const handler = async (req) => {
  verifyEvent('sha256Verifier', {
    event: req,
    secret: process.env.GITHUB_APP_SECRET,
    options: { signatureHeader: 'x-hub-signature-256' },
  })
  // rest of the handler ...
}

Let's also check that the event type is a pull request.

import { createHmac } from 'crypto'

const signRequestBody = (secret, body) =>
  'sha256=' + createHmac('sha256', secret)
  .update(body, 'utf-8')
  .digest('hex')

export const handler = async (req) => {
  const theirSignature = req.headers['x-hub-signature-256']
  const ourSignature = signRequestBody(process.env.GITHUB_APP_SECRET, req.body)
  if (theirSignature !== ourSignature) {
    return {
      statusCode: 401,
      body: 'Bad signature'
    }
  }
  const eventType = req.headers['x-github-event']
  if (eventType !== 'pull_request') {
    return { statusCode: 200 }
  }
  const event = JSON.parse(req && req.body  || '{"installation": {}}')
  return {
    statusCode: 200,
  }
}

If it's not a pull request we're returning a health status code of 200 as there's nothing wrong with the inbound request, it's just we're only interested in pull_requests.

Now that we've handled security and checked for pull request events, let's try to comment on newly opened pull requests. We're going to use Github's official npm packages for this -- 'octokit'. Go ahead and install it with 'yarn workspace api add @octokit/app'. If you're not familiar with Redwood or yarn workspaces, 'workspace api' in the previous command is just telling yarn to install this package for the backend API, and not for the web app.

Now we can initialise octokit with all the values in our '.env' file. Add the following close to the top of the file:

import { App } from '@octokit/app'

const app = new App({
  privateKey: process.env.GITHUB_APP_PRIVATE_KEY,
  appId: process.env.GITHUB_APP_ID,
  webhooks: {
    secret: process.env.GITHUB_APP_SECRET,
  },
})

Now that we've set up octokit there are a multitude of things we can do.

Finding docs

But we're aiming specifically to leave a comment on a pull request. So how do we find the docs we're after? We can start here and use the search input on the top right hand side of the page. "create comment pull" sounds like a reasonable search phrase.

The third result relates to their API so let's go there. Straight away it informs us that we're actually after the issues API with an explanation as to why and a link to the API.

More specifically, we're interested in the 'Create an issue comment' end point. Along with a great deal of information about the endpoint, it gives us the path of this endpoint "POST /repos/{owner}/{repo}/issues/{issue_number}/comments". We can use this text directly with one of the octokit helpers; it will look something like:

octokit.request(
  'POST /repos/{owner}/{repo}/issues/{issue_number}/comments',
  /* other params */
)

But before we get ahead of oursevles, let's make a function 'writePullRequestComment' that takes the web hook event, and our comment text -- i.e. it will be called like so "writePullRequestComment(event, 'Salutations, what a fine PR you have here.')".

Even though we have already initialised octokit with our '.env' secrets, there's one more step to the initialisation for setting it up for a specific installation. This is because Github needs to know the specific instance of the app that's leaving a comment. When we installed the app on our profile, this created a new instance of the app. Because this will change each time the web hook is called we do this initialisation in our new function and we can get the installation id from the event.

const writePullRequestComment = async ({ event, message }) => {
  const octokit = await app.getInstallationOctokit(event.installation.id)
  /* more code to come */
}

Now we can finish off the function by adding the request helper:

const writePullRequestComment = async ({ event, message }) => {
  const octokit = await app.getInstallationOctokit(event.installation.id)
  return octokit.request(
    'POST /repos/{owner}/{repo}/issues/{issue_number}/comments',
    {
      owner: event.repository.owner.login,
      repo: event.repository.name,
      issue_number: event.number,
      body: message,
    }
  )
}

It's now ready to be used in our handler function. Here's the handler in full:

export const handler = async (req) => {
  const theirSignature = req.headers['x-hub-signature-256']
  const ourSignature = signRequestBody(process.env.GITHUB_APP_SECRET, req.body)
  if (theirSignature !== ourSignature) {
    return {
      statusCode: 401,
      body: 'Bad signature'
    }
  }
  const eventType = req.headers['x-github-event']
  if (eventType !== 'pull_request') {
    return { statusCode: 200 }
  }
  const event = JSON.parse(req.body)
  if (['reopened', 'opened'].includes(event.action)) {
    await writePullRequestComment({event, message: 'Salutations, what a fine PR you have here.'})
  }
  return {
    statusCode: 200,
  }
}

We also added an if statement so that the bot only leaves the comment when a pull request is opened or re-opened, and not every single pull request action. Here's the result:

We've now got a mimimum example working of communication both ways between Github and our own serverless function. We're recieving a web hook from Github, and using that to take an action on Github by leaving a comment (and we verified the web hook too).

I hope you build something awesome.

Before I go I'll mention that if you're a typescript fan, then the package '@octokit/webhooks-types' is very handy. The complete example repo is typed and here's a typed version of the code:

import { createHmac } from 'crypto'
import { App } from '@octokit/app'
import type { Endpoints } from '@octokit/types'
import type { PullRequestEvent } from '@octokit/webhooks-types'

const app = new App({
  privateKey: process.env.GITHUB_APP_PRIVATE_KEY,
  appId: process.env.GITHUB_APP_ID,
  webhooks: {
    secret: process.env.GITHUB_APP_SECRET,
  },
})

const signRequestBody = (secret: string, body: string): string =>
  'sha256=' + createHmac('sha256', secret).update(body, 'utf-8').digest('hex')

const writePullRequestComment = async ({
  event,
  message,
}: {
  event: PullRequestEvent
  message: string
}): Promise<
  Endpoints['POST /repos/{owner}/{repo}/issues/{issue_number}/comments']['response']
> => {
  const octokit = await app.getInstallationOctokit(event.installation.id)
  return octokit.request(
    'POST /repos/{owner}/{repo}/issues/{issue_number}/comments',
    {
      owner: event.repository.owner.login,
      repo: event.repository.name,
      issue_number: event.number,
      body: message,
    }
  )
}

export const handler = async (req: {
  body: string
  headers: {
    'x-hub-signature-256': string
    'x-github-event': string
  }
}) => {
  const theirSignature = req.headers['x-hub-signature-256']
  const ourSignature = signRequestBody(process.env.GITHUB_APP_SECRET, req.body)
  if (theirSignature !== ourSignature) {
    return {
      statusCode: 401,
      body: 'Bad signature',
    }
  }
  const eventType = req.headers['x-github-event']
  if (eventType !== 'pull_request') {
    return { statusCode: 200 }
  }
  const event: PullRequestEvent = JSON.parse(req.body)
  if (['reopened', 'opened'].includes(event.action)) {
    await writePullRequestComment({
      event,
      message: 'Salutations, what a fine PR you have here.',
    })
  }
  return {
    statusCode: 200,
  }
}

Gotchas when using AWS docker-lambdas with the Serverless framework

Fri, 30 Apr 2021 14:00:00 GMT

I've recently spun up some lambda-docker containers for a CadHub feature. Infinitely scaling lamdas where I can control the environment sounds perfect right! The devil is always in the details and here are some sticking points that I hit.

Building on top of amazon's images is hard.

AWS needs some of their own magic on the docker images in order for them to spin them up quickly. If you build on top of one of their provided images this should be handled for you. Since I'm a frontend developer without tonnes of linux experience I found it difficult to figure out how to install the OS level packages I needed on these specialised images. I instead ended up using something more standard and easy to work with (Ubuntu) and then install Amazon's utilities on top. I found this to be easier, if you're looking to do this yourself these lines in my docker file and this `entrypoint.sh` might be of use.

Serverless CORS config doesn't agree with docker-lamdas

Using the Serverless framework, adding CORS to an endpoint should be simple and one line of config. Unfortunately, this seems more catered to normal lamdas, and found it didn't work for my docker-lamdas and ended up making options endpoints to roll my own CORS for browser-preflight.

Binary types have quirks

If you want to return binary types from your lambda you need to add API Gateway settings to allow this, but what you actually return from the lamdba is base64 encoded, but another quirk is that the input into the lambda is also base64 encoded and as far as I know this can't be turned off. The biggest drawback I found to this workflow is how much of a pain it is to test the lambda locally since you can't send it a dumby payload without first encoding it. Another thing to be aware of is that the API Gateway has a response limit of 10mb so it might be worth skipping the binary types altogether and instead dump the asset in S3 and return a signed S3 url.

Wrap up

These are all manageable problems, the worst part about it is how much they can stifle momentum when you hit unintuitive issues. Because I was trying to use node on these containers, I found most node resources to be focused on node-lamdas, not docker, so community blog posts were lacking. Would love to hear issues other front-end developers hit trying to use AWS and serverless, you can @ me.

Curated Code CAD

Fri, 30 Oct 2020 13:00:00 GMT

What is Code-CAD?

It's software that allows you to define 3D CAD models with code. It's a niche popular amongst software devs for obvious reasons — it gives you parametric models almost by default and it's easy to maintain and extend models within a team over time when paired with git. The coding nature of it allows teams to build their own abstraction for re-use and quick prototyping. The Cadhub homepage has a good breakdown of the potential of the Code-CAD paradigm. Code-CAD is not to be confused with 3d geometry libraries, Code-CAD instead has opinionated abstractions for quickly developing mechanical parts.

Which one should you use?

I recommend reading through the entire list below to see if one chimes with you and your needs, beyond that I can make the following recommendation and points:

My main recommendation is to use one of the packages that wrap OpenCascade (a mature C++ CAD library). Packages that do so are CadQuery, CascadeStudio, DeclaraCAD and pythonOCC. My reasons for recommending these are as follows:

Most of Code-CAD tools are plagued with a CSG mindset (that is unions, subtractions and intersections of primitive shapes; cubes spheres etc). This is an inherently limited paradigm (one simple example of this is how internal fillets, which are important for reducing stress concentrations in parts, become very difficult). While CadQuery, CascadeStudio, DeclaraCAD and pythonOCC still offer CSG functionality, you're also able to move beyond it with concepts like lofts and sweeps.
OpenCascade uses a B-rep (boundary representation) kernel, In my opinion, this means you'll be learning a future-proof tool that won't limit the types of applications you can model for, which is likely the case for mesh kernels, which will cause trouble in for some applications like optics and injection moulding.

OpenSCAD is tried and true, with lots of examples and tutorials floating around the internet. It also has a very intuitive syntax that many people without prior programming experience have been able to quickly pick up. However, some reasons you might want to look elsewhere are:

It can be hard to build powerful abstractions since they've rolled their own language. Consequences of this include that it doesn't have a package manager like many modern languages, and the presence of quirks with the language, such as function definitions that aren't ergonomic.
Performance can start to suffer with complex parts.
Its mesh-based kernel has limitations if you want to move beyond 3d-printed parts.

Check out the birdhouse example, while anecdotal, seeing the same part made with three different tools might help you decide which syntax you like the most.

You might want to simply pick a tool based on your language of choice. Clojure, Enaml, Go, Haskell, Lisp, Javascript and Python are all represented below.

If you want to make 3D art, Curv is specifically trying to hit that niche.

No matter which one is your tool of choice, if you're here and you love Code-CAD and you'll want to checkout Cadhub. Think of it as Codepen crossed with a thing repository, and it's our love letter to the Maker community. Currently, CascadeStudio is the only Code-CAD integration, but we're working on more. Site, repo.

Special mention

OpenScad

Repo
Community
Docs
License: GPL-2

The rest of the packages are in alphabetical order, but OpenScad gets a special mention because it's the OG. Many of the projects below were inspired by OpenScad and is the most well-known choice. If you're new to code-cad this is the safest choice. The syntax is easy to pick up and there lots of guides around the internet.

OpenCascade

Repo
Community
Docs
License: LGPL-2.1

It's a c++ library that a number the projects below wrap. OpenCascade uses a Boundary representation (B-rep) kernel, which is a powerful way representing solid geometry, this is a point of difference for some many of the other projects that use a polygon mesh.

Contributing

There are a couple of ways you can help:

Know of a package that we missed? tell us with an issue or open up a PR.
Contribute a birdhouse design in one of the tools that are missing.
Do you think we missed an important point for one of the projects, suggest more details.

Here they are:

AngelCAD

Repo
Community
Docs
License: GPL-2 or GPL-3
Online editor

AngelCAD aim to do two things:

Offer an embedded, but general-purpose scripting language for Constructive Solid Geometry, via AngelScript. This allows for a natural programming style with true variables, user-defined functions and even classes. Programmers should feel at home. See AngelCAD sample scripts.
Offer a fast boolean engine, which is powered by Carve is used for this purpose. This means that AngelCAD is generally many times faster than other mesh-based systems.

AngelCAD is capable of running OpenSCAD script for interoperability and has features like text support and DXF import coming soon.

bitbybit

bitbybit is both a node editor and Code-CAD as they have exposed a typescript interface that can be used in their app.

CadHub

A community hub for sharing code-cad projects. Currently integrates with the excellent CascadeStudio. Built and maintained by yours truly.

CadQuery

Repo
Community
Docs
License: Apache, 2.0

CadQuery is a Python library that wraps and extends OpenCascade. It has a philosophy of capturing design intent. The API has been designed to be as close as possible to how you’d describe the object to a human. An example of this is its ability to "select" parts of the model's geometry to run operations on, such as the following code that selects only the edges running along the Z-axis and fillets them.

result = cq.Workplane("XY" ).box(3, 3, 0.5).edges("|Z").fillet(0.125)

CascadeStudio

Repo
Community
No official docs, but the online editor does have IntelliSense support
License: MIT
Online editor

A javascript wrapper for OpenCascade that runs in the browser. (OpenCascade can run in the browser when compiled to web-assembly). CadHub integrates with CascadeStudio.

Curv

Repo
Community (mailing list)
Docs
License: Apache, 2.0

Curv is a programming language for creating art using mathematics. It’s a 2D and 3D geometric modelling tool that supports full colour, animation and 3D printing. It was inspired by OpenScad and shadertoy.

FreeCAD

Repo
Community
Docs
License: LGPLv2

FreeCad is a more traditional CAD package that supports python scripting, Both for modelling as well as controlling the FreeCAD GUI itself. Not only that it has a built in OpenScad workbench as well as an external CadQuery workbench, making it the best in this list at interoperability. FreeCAD uses OpenCascade under-the-hood.

ImplicitCAD

Repo
Docs
License: AGPL-3
Online editor

Inspired by OpenScad with a very similar language, implemented in Haskell and includes the ability to write definitions in Haskell, instead of just OpenSCAD, and is part of an 'almost stack' of tools including ExplicitCAD (for a GUI), and HSlice (for an STL slicer).

DeclaraCAD

Repo
Community
Docs
License: GPL-3
Online editor

A declarative parametric 3D modelling program built using OpenCASCADE and enaml.

JSCAD

Formerly OpenJsCad, also inspired by OpenScad. The project is close to a v2.0 release. Implemented in javascript allowing it to be modelled directly within a browser.

libfive

Repo
Community (Github Issues)
Docs
License: Mozilla Public License 2.0 and GPL-2 or later

Libfive is a software library and set of tools for solid modelling, especially suited for parametric and procedural design. Lisp based language, (so (you (((((can expect ) lots of parentheses))))).

pythonOCC

Repo
Docs
License: LGPL-3
Online editor

Python-based, Also uses OpenCascade.

RapCAD

Repo
License: GPL-3

Another project inspired by OpenScad. The author considers key differences to be procedural vs functional programming language style, (i.e variables can be modified) and the use of arbitrary precision arithmetic throughout (meaning there are no unexpected double/float rounding errors). There is a handy feature matrix between RapCAD, OpenScad and ImplicitCad.

scad-clj

Repo
No docs but mirrors openscad functions
License: EPL-1.0

OpenSCAD DSL in Clojure. Functions generally mirror OpenSCAD, with a couple of notable exceptions.

scad-hs

Repo
No docs but mirrors openscad functions
License: BSD-3-Clause License

Same author as scad-cji, he likes functional programming languages clearly.

sdfx

Repo
Community
Docs
License: MIT

Go-based Code-CAD package that uses a signed distance functions (SDFs) kernel. Is capable of doing fillets and chamfering. The repo includes a standard-library.

SolidPython

Repo
Docs
License: GPL-2 or later

Python-based library that wraps OpenScad, i.e. it outputs OpenScad code.

Tovero

Repo
Community (Gitlab Issues)
Docs
License: LGPL-2.1 or later and GPL-2 or later

Tovero is a binding of Libfive to Common Lisp, including a standalone REPL-based viewer. Tovero can be integrated with Clive, a Common Lisp scene graph and 3D GUI, to build more complex modelling applications.

Node editors / other

Not quiet Code-Cad, but they do embody much of the same thought process.

BlocksCAD

Look unmaintained.

Dynamo

Repo
Community
Docs
License: Apache 2.0

Dynamo is, quite literally, what you make it. Working with Dynamo may include using the application, either in connection with other Autodesk software or not, engaging a Visual Programming process, or participating in a broad community of users and contributors. Works with FreeCad.

MakeCode

Repo
Docs (tutorials)
License: MIT
Online editor

MakeCode's block editor supplies many helpers that make it perfect for making functional 3d prints, for example, there are functions that help stack and layout parts, as well as fillet utils (called styled edges). It also has a fast hull function (called wrap shapes). MakeCode is sponsored by BuildBee.

Sverchok

Repo
Docs
License: GPL-3

Add-on for blender. Sverchok is a powerful parametric tool for architects, allowing geometry to be programmed visually with nodes.

Something I missed? Open an issue on Github

Testing netlify identity functions within RedWoodJS

Tue, 20 Oct 2020 13:00:00 GMT

So your working on a RedWood app and have opted to use netlify identity. That's great, allowing netlify to do the heavy lifting of your Auth, while still letting you hook into particular events by triggering "identity events". This is covered a little in the RBAC cookbook, but here I want to talk specifically about using these events to write to your db, such as creating a new user. Here's an example.

// src/functions/identity-signup.js
import { createUser } from 'src/services/users/users.js'
export const handler = async (req, _context) => {
  const body = JSON.parse(req.body)
  const user = body.user
  const email = user.email
  if (body.event === 'signup') {
    await createUser({input: {
      email,
      bio: 'default bio'
    }})
  }
  return { statusCode: 200 }
}

We're importing one of our services, but of course db can be imported directly. Now that we're ready to test our code we want to avoid deploying for the sole purpose of testing it, because even if you have a staging environment setup where test data can be created, it's still a slow development cycle and burns up build minutes.

First a point of clarification, by testing, I mean manual verification checks done while in development, not automated tests, though some of this information could be leveraged for that too.

We can use netlify's cli to invoke these functions. Detailed information is on the netlify community forum, but in short the steps are:

# Install the cli
yarn add netlify-cli -g

# Rebuild api after your last changes to /functions
yarn rw build api

# Invoke your function with the CLI, pointing it to the rw dev port
netlify functions:invoke identity-signup --port 8910

Netlify will invoke the function and include relevant test data as if it were being called in production. For example here's the body it passed through.

{
  event: 'signup',
  user: {
    id: '1111a1a1-a11a-1111-aa11-aaa11111a11a',
    aud: '',
    role: '',
    email: 'foo@trust-this-company.com',
    app_metadata: { provider: 'email' },
    user_metadata: { full_name: 'Test Person' },
    created_at: '2020-10-21T09:43:23.992Z',
    update_at: '2020-10-21T09:43:23.992Z'
  }
}

The reason you need to rebuild the api after each change is because the netlify-cli uses your netlify.toml to determine where the functions are, and this should point to your build directory `functions = "api/dist/functions"`

You should now see changes you expect to data in your local db.

bending time to our will

Tue, 06 Oct 2020 13:00:00 GMT

What if I told you that humble clock was alive, and has not only the power but also the motive to:

- shrink the oceans

- pull down the heavens

- make lots of money

- and perhaps, stab you in the back

I'm referring to the clock generally, but the device that I'm holding is very much a product of the maker movement. It's 3D printed, the creator put this out for free, and one thing I appreciate is he's used his page to draw attention to horology - the craft of clock making.

The reality is though that in the context of today this is nothing more than a toy. It's good to have some reminders of the past around us. But the device alone can't tell you its tale.

We're going to use the clock today to learn something about emerging technologies. The clock is useful because it's origin is unknown to most, which allows us to reflect upon it with fresh eyes, without bias. They say that retrospect is 20/20, but that's not always accurate depending on the context, if history is a road with many forks, where a single path is taken at each split. Then, reflecting back gives us the illusion of a single path since we can no longer see all the forks. Take a well-known piece of transformative technology, such as Gutenberg's press. You're probably not an expert on the matter, but likely know that by making text and books easily and cheaply reproducible, the printing press led to an explosion of ideas impacting many parts of European society and then the world. Because it's better known, it easily provokes a well-trodden narrative about the technology, like the one I quickly mentioned. Such a narrative makes it hard for us to imagine other possibilities for the technology and makes it feel like both the invention and its subsequent impact was inevitable and I would argue that's not the case. For one thing printing presses already existed before Gutenberg's improvements and there were many cultural forces at play that contributed to the impact that the press had. Plus the impact it would have was not predicted at the time.

Back to clocks, how did this tech shape society? The full history of all time-related devices such as sundials and hours glasses goes way back. Though a watershed moment for clocks was the invention of the escapement, which is the mechanism in the clock that gives it it's precise ticking.

What's important, though, is who invented it and what their motivation was. By who I mean which group of people as we don't know the individual, instead we know it was twelfth-century Benedict monks trying to improve their timekeeping as they had time allotments for many things that made up life in the monastery. Being monks, times of devotion was significant to them, and in fact, they had seven periods of devotion a day. It's understandable that better precision in timekeeping would help with their devotions, rather than relying upon one of the monks ringing a bell multiple times a day to prompt their rituals.

Once the escapement, hence the clock hand, had been invented it soon "escapemented" the monastery walls and by the mid-fourteenth century there were many "town" or "public" clocks. The roll-out of the clock did not only increase the precision of timekeeping but changed the notion of time itself. Conical hours was the time kept by the church and it was pretty rough because it never needed to be exact. For example, vigil was from about 2 am to dawn, matin was just a later portion of vigil, lauds was dawn and so obviously would vary with the seasons and so on and so forth. Now with an exact measurement of time possible, suddenly time was no longer a sequence of events that followed on from one another but instead, a resource that should be spent carefully. A sign of this is what was engraved on many of the early time-pieces like "time is short", "wasting time" etc. Phrases we're familiar with, but we don't feel the need to write onto our clocks. Such a change in society's understanding of time, obviously had social consequences.

But first who pushed for these "town" clocks? It was the merchant class, as timekeeping fit in well with business, they also enjoyed that the new time standards gave separation from the church. It's not surprising that Italian cities like Venice and Genoa saw the largest boom in clocks since they were famous for trade, and the lives of businesspeople became more regimented than ever.

The church wasn't happy about this. One simple reason is that there were now conflicts in the ringing of bells between the church's time and the town clock's time. As the merchant class gained influence and secular life grew, more often than not the church was forced to change its bell timing, not the reverse. There were more theological reasons why the church didn't like the advent of the clock. They were concerned with the public's new conception of time as a challenge to their faith, as time previously was thought to be part of nature and so belonged to God. The new, more abstract view of time changed that, so now it belonged to the individual, as merchants started to sell their time directly in hour increments instead of days, it was seen as selling something that belonged to God. The church fought the clock, but ceded ground and the clock is here to stay.

The clock became a force for controlling and synchronizing the actions of workers. By introducing regular work hours, it also helped pave the way for the industrial revolution and the rise of capitalism centuries later.

To bring us back to the silly thought that the clock is alive and has its own motivations. I'm not serious about the alive part obviously, but it is accurate that technologies have particular bias embedded in them that can exploit characteristics of the culture they find themselves in, often against the best wishes of the inventor. Devote monks set out to produce a device that would help their spiritual pursuits and ended up, instead being most efficient at aiding in monetary activities.

At the same time, the clock shrunk oceans by aiding in navigation, pulled down the heavens with its uses in astronomy, made money for the rising merchant class, and it betrayed its original inventors by reducing spiritual life.

During the Reformation, a period that saw the catholic church lose its monopoly status over spiritual life in Europe, Martin Luther, a prominent figure in these events, understood what having a bible in every home would mean for the catholic church's power, as, if everyone could read God's word for themselves they wouldn't need the churches hierarchies to be the arbiters of spiritual truth. That is to say, Luther understood and exploited an inherent bias in the technology of the printing press.

Which brings us back to Gutenberg, a devote catholic, who would have turned in his grave if he learned the way Martin Luther was using his device. Another example of technology betraying its creator.

It seems that currently there are only two, polarised views on tech. There's a camp that is extremely critical and spends all their time imagining dystopias, and the other that equates technical progress with human progress.

We need a more complex and nuanced view of tech, as we are currently facing a big problem with misinformation and there's a comparison to be made between the printing press and the internet. After the printing press was invented and dramatically reduced the cost of information, there was a lot of tension between different religious factions (Catholic and protestant). It started with violent language and eventually, outright violence, in fact about 130 years of it. With the advent of the internet just a handful of decades ago, we've seen a dramatic reduction in the cost of information and also rising political tensions with lots of violent language. I sure hope that we don't have another 130 years of violence, but I'm not terribly hopeful given the kinds of solutions that are being proposed. Take Twitter's "bird watch" program, which is a bandaid solution if I've ever seen one. It seems very little people are actually proposing to ban some of the root causes, which seem to be recommendation algorithms that optimise for engagement but have the side effect of promoting false information.

That's a massive topic so I'll stop there, but I'd recommend a talk by Niall Ferguson https://www.youtube.com/watch?v=07KKYostAJ0

As a closing statement, I'd suggest you try and get less screen time. Remember that there are whole teams of engineers trying to get you to spend more time staring at your device, and it's not in your best interest.

OpenScad Review - Is it worth learning?

Sat, 05 Sep 2020 14:00:00 GMT

This is an:

Overview of OpenScad and where it fits in with other cad packages.
What's it good for and not so good for,
And whether I think it's worth learning.

Checkout out the video version here:

OpenScad is a CAD or computer-aided-design package. In a nutshell, it's software for making 3d models that are dimensional and mechanical in nature, as opposed 3d graphics software like blender, which is better suited for artistic endeavours like assets for a game, as an example.

Since OpenScad is cad software, it's amongst packages like fusion 360, Inventor, Freecad, Onshape and many more. The point of difference though for OpenScad, is that it's all programmed. Every part of the model comes from text you write, and the GUI part of the software is only there to inspect your model.

This is worlds apart from the click and drag nature of the other packages. The programmed nature of OpenScad gives it some unique advantages that have resulted in somewhat of a cult following for the software, but also some severe drawbacks.

Let's start with its pros.

1) It's Opensource

OpenSCAD is free, open-source software that you can download and run on your machine and do whatever you want with. You don't need an account; there's no vendor lock-in or licensing issues. Also OpenScad files are likely very robust and will continue for work for many years.

2) Community libraries

Being opensource means there's a great open-source community with many libraries available that can help make you more productive or open up new possibilities.

3) Fostering community

Parts being 100% code is a significant benefit to the community, as hosting code on Github is a very robust way of making community contributions, as it goes through the same process software would, without awkwardly trying to pass around save files when changes are made.

4) It Hits a niche

OpenSCAD is very intuitive for those that come from a programming background.

5) Power of a programming language

There are some creative and straight-up wacky things you can do when you have a programming language at your disposal that otherwise wouldn't be possible. Also because it's all code, there are creative ways that it can be fed into a pipeline. A real-world example is Thingiverse's customizer that allows you to edit variables for Openscad models uploaded to the site and download the resulting STLs.

6) Parametric to the core

Most cad packages these days are parametric. Loosely that means that as they're modelled, they're given dimensions along the way that can be changed later and have the model adjust itself. An example might be a bearing housing that's made for a particular sized bearing, if you want to bump up the size by a few millimetres it shouldn't be a problem, but as you can imagine there would be limits to this. If the bearing is made twice the size of the rest of the model, then the resulting shape is probably not going to work if it's not out-right broken. This is true for OpenScad too, though, with the power of a programming language, you're able to put whatever logic into the model to help it deal with extreme values if that's needed. Going so far to completely re-arrange the model to make things work if need be.

7) Beginner-friendly language

The language is fairly simple and therefore easy to pick up. This makes it a friendly first programming language for those looking to learn. Learning a skill that can outlive your use of the software is a great side-benefit. And the visual nature of the code results in rapid and satisfying feedback. Couple that with the fact you're making solid models that can be 3d printed and you get an experience that's tangible, I mean physically tangible in ways that other languages can't compete with. That makes it particularly compelling for getting kids into programming, which I think is fantastic.

Now for the cons:

1) Unintuitive CAD interface

The fact is that for most people, coding up a part is not intuitive, because 3d modelling is inherently very visual, a graphical interface is better, at least for getting started.

2) Complex shapes can be challenging

This largely stems from the fact that the core process of making parts in OpenScad is with boolean operations of primitive shapes, that is union, difference and intersection. This paradigm of modelling doesn't lend itself particularly well to smooth transitions, flowing curves or even something trivial in other packages like adding a fillet is difficult. This can be somewhat alleviated with the help of other libraries, but it's still a firm con of the software.

3) Time-consuming

This follows on from the last point as they are connected, the difficulty of modelling complex parts means more tinkering is needed to get what you what.

4) The language is limiting

It might seem ironic that I'm now criticising the language since I put it in the pro list for being easy to pick up, but that's the flip side of its simple nature. OpenScad is it's own language and generally, I think rolling your own language for a project is treacherous. On the one hand, you might be able to design the language to meet the specific needs of the project very well. But on the other hand, you have to re-invent the wheel for so many things that are common to all programming languages. Effectively, the project is building two things at once that it has to get right, their software and a language. Further, if it had instead been built on top of another language then it would also be possible for users to take advantage of libraries that already exist, say math libraries. In fairness, if the project had been built on another language, it would likely make it less beginner-friendly. Because even if beginners only needed to know a small subset of the language to get started, They could still easily get lost in all of the existing documentation for that language. I think my biggest problem with OpenScad rolling their own language is without a lot of resources it's inevitable that the language would be sub-par with what developers are used to. Sure OpenScad is bound to have fewer features than other languages, that's fine but even what has been implemented feels a little janky. Functions aren't intuitive and defining scoped variables within them feels janky. For-loops are weird too, and re-assigning variables doesn't cause errors but easily lead to bugs as it's behaviour is counter-intuitive.

5) Lacking mature features

There are features that OpenScad lacks like FEM or CAM. but I'd label these as specialised features and not mature features, as it's not like every CAD package includes these, and it's reasonable for OpenScad to focus just the modelling stage of a part's lifecycle. My point is simply it's not a big deal these feature's aren't there, and I don't think OpenScad suffers because of it. What's more of a problem though is limited file format compatibility. OpenScad's lack of support for STEP files in particular, as they are an industry standard for manufacturing. If you want to move beyond 3d printing and start getting parts made with injection moulding, then you might need to redo parts in another package. From what I understand, STEP files are very difficult for OpenScad for historic reasons. Something worth considering though under the theme of "mature features" is "are there companies using OpenScad?" Hard to say but I certainly don't know of many. Prusa did use OpenScad for their printers, in fact all the scad files for their mk2 printer are up on github. Though if you look at their latest printer's github (prusa-mini), the files are all .ipt or inventor files. That doesn't mean they never use OpenScad now, but it is a sign that they've found it prudent to move away from it somewhat.

Closing thoughts

With all of that out of the way, my answer to the question of should you learn OpenSCAD my short answer is yes, and my long answer is it depends, and the best way for me to break down the long answer is to propose some scenarios.

If you are a programmer then go ahead, you'll probably enjoy it.
If you are interested in learning to code, then OpenScad is a pretty gentle introduction. There are plenty of programming concepts that you will never learn in OpenScad, but they can come later with the next language.
If you want to make parts that are very robust to changing parameters and want to be able to host it on Thingiverse, where users can put in their own parameters, then OpenSCAD is the only option available.
If you like the concept of OpenSCAD, whether that's the code, fostering community contributions, or lack of vendor lock-in, and you are making parts to be 3d printed, then OpenSCAD is a solid choice.

However, if you need to be productive today, or need to support STEP files for manufacturing, or need to make very complex shapes with smooth flowing meshes, then you'll be battling against OpenScad to get it to do what you need and would be a poor choice.

Some other things to consider before you make your choice are:

If the lack of vendor lock in, and opensource nature is appealing to you, then you could always consider other open-source CAD packages like FreeCad.
If you love the idea of programming-CAD, while OpenScad is the most popular, there are alternatives. SolidPython is a python wrapper for OpenSCAD, or one of my favourites is jsCAD. It's implemented in javascript, and models can be made right in the browser.
If you're worried about what I said about it being difficult to make complex shapes, have a look at libraries that are available first, as it should give you a good idea of the kinds of parts you can produce. The libraries officially recommended by OpenScad are an excellent place to start, but if I may, I'll plug my library, Round-Anything. The library has a variety of features, but what I think is of most significance is offering a way to add rounding to polygons. The result is a move away from the "boolean with primitives" paradigm, towards the sketch and extrude paradigm, which is, in my opinion, the workhorse of more traditional cad packages.

One interesting take-away from everything I've said thus far is that even with some pretty heavy criticism of the software, I still recommend it for a variety of situations. This speaks to how well OpenScad fills a niche, and I think that means that the software will continue to be used and loved for many years to come.

Round-Anything API

Sat, 01 Aug 2020 14:00:00 GMT

This is a deep dive into the Round-Anything library API for OpenScad.
For a general overview of features and how to get started and the motivation behind the library, see the written overview or the video overview below.

Quick side-note

I'm currently working on a community website for "Code-CAD" (like OpenSCAD). A good way to think of it is codepen crossed with a thing repository. You can check it out at cadhub.xyz or it's repo.

MinkowskiRound

This module will round its children. External and internal radii can be defined separately. The syntax is:

minkowskiRound(OR, IR, enable, boundingEnvelope) {
  // children
}

OR: Set the external radius
IR: Set the internal radius
enable: Toggle whether the rounding enabled, as the module is computationally expensive, so it's convenient to have an easy way to disable it.
boundingEnvelope: An array with three values which should be large enough to capture the children

Because this module will round anything after-the-fact, it's useful adding radii to complex shapes that would be very difficult to do otherwise. Such as the curving internal edge of this cube-cylinder union.

$fn=20;
minkowskiRound(0.7,1.5,1,[50,50,50])
  union(){
    cube([6,6,22]);
      rotate([30,45,10])
        cylinder(h=22,d=10);
  }

polyRound

Function for adding radii to any point of a polygon. Syntax is:

polygonArray = polyRound(radiipoints,fn,mode)

radiiPoints: nest array of [x, y, r] points. That is x-y coordinates and the radius for that point, .ie. [[x1, y1, r1],[x2, y2, r2] ...]
fn: The amount of point each radius is subdivided.
mode: Three different modes for handling conflicting radii:

Default, automatically reduces radii to stop conflicts.
Debugging mode, print reduced radii to the console.
Radii conflict resolution disabled.

The function only returns the polygon points, therefore it's typical to pair it with polygon and linear_extrude. Here's a simple example

radiiPoints=[[-4,0,1],[5,3,1.5],[0,7,0.1],[8,7,10],[20,20,0.8],[10,0,10]];
polygon(polyRound(radiiPoints,30));

To further understand how the radii conflict resolution works here's an example of that.

//example of radii conflict handling and debuging feature
function makeRadiiPoints(r1, r2)=[[0,0,0],[0,20,r1],[20,20,r1],[20,0,0]];

// the squre shape being 20 wide, two radii of 10 both fit into the shape (just)
translate([-25,0,0])polygon(polyRound(makeRadiiPoints(10,10),50));

//radii are too large and are reduced to fit and will be reduce to 10 and 10
translate([0,0,0])polygon(polyRound(makeRadiiPoints(30,30),50));

//radii are too large again and are reduced to fit, but keep their ratios r1 will go from 10 to 4 and r2 will go from 40 to 16
translate([25,0,0])polygon(polyRound(makeRadiiPoints(10,40),50));

//mode 2 = no radii limiting
translate([50,0,0])polygon(polyRound(makeRadiiPoints(15,20),50,mode=2));

See bellow for an even deeper dive into the logic behind it.

translateRadiiPoints

Function for moving radii points to make their reuse easier. Syntax is:

translatedRadiiPoints = translateRadiiPoints(radiiPoints, tran, rot);

radiiPoints: list of [x, y, r] points to be translated.
tran: [x, y] translation points.
rot: how to rotate the points on the z-axis.

Because the function returns radiiPoints, they still need to be passed to the polyRound function before used as a polygon. Though realistically points are used multiple times in one part hence the need to translate them while still in point form, and so would be combined with concat first. Here's a simple example.

nutW=5.5;   nutH=3; boltR=1.6;
minT=2;     minR=0.8;
function nutCapture(startAndEndRadius=0)=[
  [-boltR,        0,         startAndEndRadius],
  [-boltR,        minT,      0],
  [-nutW/2,       minT,      minR],
  [-nutW/2,       minT+nutH, minR],
  [nutW/2,        minT+nutH, minR],
  [nutW/2,        minT,      minR],
  [boltR,         minT,      0],
  [boltR,         0,         startAndEndRadius],
];
translate([-5,0,0])polygon(polyRound(nutCapture(),20));

negativeNutCapture=translateRadiiPoints(nutCapture(),tran=[5,0]);
rotatedNegativeNutCapture=translateRadiiPoints(nutCapture(1),tran=[20,5],rot=90);
aSquare=concat(
  [[0,0,0]],
  negativeNutCapture,
  [[20,0,0]],
  rotatedNegativeNutCapture,
  [[20,10,0]],
  [[0,10,0]]
);
polygon(polyRound(aSquare,20));

shell2d

Module that will create a shell out of any 2d object. If given more than one child, the inside will be filled with the following children. Syntax is:

shell2d(offset1,offset2=0,minOR=0,minIR=0){
  // shell child
  // fill children
}

offset1, offset2: Two offsets that together define the thickness of the shell and are measured relative to the perimeter of the original 2d shape. Negative value go towards the centre of the shape, positive value go away.
minOR, minIR: minimum radii can be defined, if you're using this in conjunction with ployRound they can be ignored for the most part.

Here's a simple example.

radiiPoints=[[-4,0,1],[5,3,1.5],[0,7,0.1],[8,7,10],[20,20,0.8],[10,0,10]];
shell2d(-0.5)polygon(polyRound(radiiPoints,30));
translate([0,-10,0])shell2d(-0.5){
  polygon(polyRound(radiiPoints,30));
  translate([8,8])gridpattern(memberW = 0.3, sqW = 1, iter = 17, r = 0.2);
}

beamChain

This function takes a series of radii points and creats a beam of constant thickness with each pair of points. Syntax is:

radiiPoints = beamChain(radiiPoints, offset1, offset2, mode, minR, startAngle, endAngle)

radiiPoints: radiiPoints: list of [x, y, r] points. Note, negative radiuses only allowed for the first and last radii points.
offset1, offset2: The two offsets that give the beam it's thickness. When using with mode=2 only offset1 is needed as there is no return path for the polygon.
minR: Min radius, if all of your radii are set properly within the radii points this value can be ignored
startAngle, endAngle: Angle at each end of the beam, different mode determine if this angle is relative to the ending legs of the beam or absolute.
mode: Different modes for how the end angles are handled and if the return path of the beam polygon in included.

StartAngle and endAngle are relative to the angle of the last two points and equal 90deg if not defined.
Only the forward path is defined, useful for combining the beam with other radii points, see examples for a use-case.
StartAngle and endAngle are absolute from the x axis and are 0 if not defined.

This function is very flexible in how it's used so below are a series of examples increasing complexity.

The first shows how a series of points can form the bean chain, how radii can be added and adding thickness to the beams

The second example shows adding a angle and filleting radius to the end of the beams

Lastly this example show how seperating the beams polygong path into forward and return paths can be used to add extra polgon points at the beam. The advantage of this over regular union is adding a transitioning radius between the beam and the extra points.

function beamPoints(r1,r2,rStart=0,rEnd=0)=[[0,0,rStart],[2,8,0],[5,4,r1],[15,10,r2],[17,2,rEnd]];

// chained lines by themselves
translate(){
  radiiPoints=beamPoints(0,0);
  for(i=[0: len(radiiPoints)]){color("red")translate([radiiPoints[i].x,radiiPoints[i].y,0])cylinder(d=0.2, h=1);}
  polygon(polyRound(beamChain(radiiPoints,offset1=0.02, offset2=-0.02),20));
}


// Add some radii to the line transitions
translate([0,-7,0]){
  radiiPoints=beamPoints(2,1);
  for(i=[0: len(ex3)]){color("red")translate([radiiPoints[i].x,radiiPoints[i].y,0])cylinder(d=0.2, h=1);}
  polygon(polyRound(beamChain(radiiPoints,offset1=0.02, offset2=-0.02),20));
}

// Give make the lines beams with some thickness
translate([0,-7*2,0]){
  radiiPoints=beamPoints(2,1);
  polygon(polyRound(beamChain(radiiPoints,offset1=0.5, offset2=-0.5),20));
}

// Add an angle to the start of the beam
translate([0,-7*3,0]){
  radiiPoints=beamPoints(2,1);
  polygon(polyRound(beamChain(radiiPoints,offset1=0.5, offset2=-0.5, startAngle=45),20));
}

// Put a negative radius at the start for transationing to a flat surface
translate([0,-7*4,0]){
  radiiPoints=beamPoints(2,1,rStart=-0.7);
  polygon(polyRound(beamChain(radiiPoints,offset1=0.5, offset2=-0.5, startAngle=45),20));
}

// Define more points for a polygon to be atteched to the end of the beam chain
clipP=[[16,1.2,0],[16,0,0],[16.5,0,0],[16.5,1,0.2],[17.5,1,0.2],[17.5,0,0],[18,0,0],[18,1.2,0]];
translate([-15,-7*5+3,0]){
  for(i=[0:len(clipP)-1]){color("red")translate([clipP[i].x,clipP[i].y,0])cylinder(d=0.2, h=1);}
  polygon(polyRound(clipP,20));
}

// Attached to the end of the beam chain by dividing the beam paths in forward and return and
// concat other polygon inbetween
translate([0,-7*6,0]){
  radiiPoints=beamPoints(2,1);
  forwardPath=beamChain(radiiPoints,offset1=0.5,startAngle=-15,mode=2);
  returnPath=revList(beamChain(radiiPoints,offset1=-0.5,startAngle=-15,mode=2));
  entirePath=concat(forwardPath,clipP,returnPath);
  polygon(polyRound(entirePath,20));
}

// Add transitioning radii into the end polygong
translate([0,-7*7-2,0]){
  radiiPoints=beamPoints(2,1,rEnd=3);
  forwardPath=beamChain(radiiPoints,offset1=0.5,startAngle=-15,mode=2);
  returnPath=revList(beamChain(radiiPoints,offset1=-0.5,startAngle=-15,mode=2));
  entirePath=concat(forwardPath,clipP,returnPath);
  polygon(polyRound(entirePath,20));
}

// Define multiple shells from the the one set of points
translate([0,-7*9,0]){
  for(i=[0:2]){polygon(polyRound(beamChain(ex3,offset1=-1+i*0.4, offset2=-1+i*0.4+0.25),20));}
}

mirrorPoints

Function for mirroring radiiPoints. The advantage of this over other mirror techniques is when using radii points it allows for adding a radius to the transition of the two halves. syntax is:

mirroredRadiiPoinst = mirrorPoints(radiiPoints, rot, endAttenuation)

radiiPoints: radiiPoints: list of [x, y, r] points.
rot: angle of rotation.
endAttenuation: [start, end]. Amount of points to be removed from either end of the radiiPoints. Its purpose is to remove single points from the ends if they lie right on the mirror axis and would cause two points on top of each other.

Below is a simple example.

centerRadius=7;
points=[[0,0,0],[2,8,0],[5,4,3],[15,10,0.5],[10,2,centerRadius]];
mirroredPoints2=mirrorPoints(points,0,[0,0]);
translate([0,-20,0])polygon(polyRound(mirroredPoints2,20));

polyRoundExtrude

Module for extruding a radiiPoints, where the top and the bottom of the extrusion can be rounded. syntax is:

polyRoundExtrude(radiiPoints,length,r1,r2,fn,convexity)

radiiPoints: radiiPoints: list of [x, y, r] points.
length: length of the extrusion.
r1, r2: Start and end radii.
fn: amount of subdivisions for forming the polyhedron.
convexity: convexity of the underlying polyhedron.

This module is very similar to extrudeWithRadius in purpose, though by using radiiPoints directly instead of a generic 2d child it's abet to offer smoother curves in a more preformant manner, and so is recommended over extrudeWithRadius where possible. Below is a simple example.

radiiPoints=[[10,0,10],[20,20,1.1],[8,7,10],[0,7,0.3],[5,3,0.1],[-4,0,1]];
polyRoundExtrude(radiiPoints,2,0.5,-0.8,fn=50);

extrudeWithRadius

Module for extruding a 2d child, with the ability to put a radius on each end of the extrusion. This module is similar to polyRoundExtrude, though this module is more flexible as it's able to work on any 2d child, however it's less preformant. polyRoundExtrude is recommended over extrudeWithRadius if your use-case allows it.
Syntax is:

extrudeWithRadius(length,r1=0,r2=0,fn=30){
  // 2d child
}

length: length of the extrusion.
r1, r2: Start and end radii.
fn: How much the radii are subdivided.

Below is a simple example.

radiiPoints=[[-4,0,1],[5,3,1.5],[0,7,0.1],[8,7,10],[20,20,0.8],[10,0,10]];
extrudeWithRadius(3,0.5,0.5,50)polygon(polyRound(radiiPoints,30));
#translate([7,4,3])extrudeWithRadius(3,-0.5,0.95,50)circle(1,$fn=30);

Round-Anything, a pragmatic approach to OpenSCAD design

Wed, 22 Jul 2020 14:00:00 GMT

Round-Anything is primarily a set of OpenSCAD utilities that help with rounding parts, but it also embodies a robust approach to developing OpenSCAD parts. I built this library to solve some of my own struggles with rounding and OpenSCAD, and I've been happy to see other's are finding it useful too.

The Why

The truth is radii, internal radii, in particular, can be a real pain to add in openscad. And the more you move away from shapes with 90 degree angles, the more difficult it becomes, effectively putting a complexity ceiling on parts you can produce in OpenScad. Because of how important radii in both making an appealing and strong part, reducing stress concentration etc. A library that focuses on radii as a core principle makes for a solid foundation for your parts. Furthermore, the heart of the library revolves around the polygon, this is because we're leveraging the battle-tested paradigm of extruding from 2d sketches of most CAD packages. I can't imagine making an OpenScad part without Round-Anything.

Below is an overview of the library's features, For documentation of its API click here.
There's also a timelapse of a part made using the library.

A video version of this article can be found below:

Quick side-note

I'm currently working on a community website for "Code-CAD" (like OpenSCAD). A good way to think of it is codepen crossed with a thing repository. You can check it out at cadhub.xyz or it's repo.

Features

Minkowski Round

A power tool for rounding all edges of a design after the fact. Adjustable for external and internal radii.

PolyRound

Gives the ability to round each point of a polygon.

Radius Conflict resolution

Can automatically Shrink radii for any pair that are too close together and would cause a conflict, can be opted out of.

Point translation helpers

Aids in the reuse of polygon of points. To keep a single source of truth and for editing them together.

2D-shell

Self-explanatory, when combined with poly round, it gives a nice smooth transitioning shell.

Beam chain

Unlike a regular polygon, where the last point is joint to the first to complete the shape. Instead, we can treat each set of points as a beam without it wrapping at the end. We can round the transitions between the beams and combine with other points, to form more complex shapes.

An offset from the normal gives beams their thickness.

Mirror points

For mirroring a set of points. The benefit of this over using the native OpenSCAD mirror transformation is the ability to put smooth radii along the mirrored plane.

polyRoundExtrude

Allows for putting a radius on the end of extrusions when using polygons. A negative vaule will flare the end out for a transition to flat.

Radius extrude

Similar to the above but will work with any 2d shape, though uses a more hacky and less proformant method.

Deep AST matching guide

Wed, 08 Jan 2020 13:00:00 GMT

So you want to take advantage of the javascript AST, and move beyond elemental use cases, (e.g. finding string concatenation in order to convert to template literals).

The technique well go over is match more than a single node, while keeping track of where we are within the AST as we traverse it. As a simple example, let's say you want to match function-returns where the return is an object with a key called `coolKey`, but only if the function name also starts with `cool`. We're going to need a rough understanding of the structure of the AST, and how AST traversing libraries move through the tree structure.

Firstly the structure of the AST. For any given thing you want to match on, you can simply paste your code into astexplorer.net and inspect it, though something to keep in mind is that it's generally safe to assume that the AST will follow the same rough structure as your code indentation. We use indentation as a way to visually represent hierarchy of the code, and the AST has a similar hierarchy (though with a great deal many more levels). They are similar enough to know that the return statement of a function is going to be nested further in the structure than the function definition.

Now AST libraries. These generally take callbacks for entering and existing nodes, though in what order? How do they pick the next node to enter? Since most folks will be familiar XML style tree structure, let's use that as an example.

```
<body>
    <node1>
        <node2>
            <node3>
                <node4>
                </node4>
            </node3>
        </node2>
        <node5>
        </node5>
    </node1>
    <node6>
    </node6>
    <node7>
    </node7>
</body>
```

It will call enter and leave nodes in the same order as if you were reading the above line-by-line. Or how I like to think of it is, once it enters a node, it will fully explore everything nested in it before it leaves that node. To be 100% explicit the order would be.

enter-node1, enter-node2, enter-node3,
enter-node4, leave-node4, leave-node3,
leave-node2, enter-node5, leave-node5,
leave-node1, enter-node6, leave-node6,
enter-node7, leave-node7

Back to our first example, putting the simplest version of what we're trying to match into AST explorer.

function coolFunction() {
    return {
      coolKey: 'should match',
    }
}

The AST for this (with properties remove for brevity) would look like:

{
    "type": "FunctionDeclaration",
    "id": {
        "type": "Identifier",
        "name": "coolFunction"
    },
    "body": {
        "type": "BlockStatement",
        "body": [{
            "type": "ReturnStatement",
            "argument": {
                "type": "ObjectExpression",
                "properties": [{
                    "type": "Property",
                    "key": {
                        "type": "Identifier",
                        "name": "coolKey"
                    },
                    "value": {
                        "type": "Literal",
                        "value": "should match",
                    },
                }]
            }
        }]
    }
}

This example is simple enough that we could match the entire thing from the function declaration node. However, that wouldn't be very robust as the return statement could be nested in an `if` statement, possibly multiple times, changing the structure of the AST significantly. It's better to match the function declaration node and the return node separately. Doing so will also be useful later if we want to expand code to cover more cases, say class and arrow function declarations.

Code for matching the function declaration and the return statement:

const isCoolFunction = node => 
    node.type === 'FunctionDeclaration' &&
    node.id.type === 'Identifier' &&
    node.id.name.startsWith('cool')

const isCoolReturn = node => 
    node.type === 'ReturnStatement' &&
    node.argument.type === 'ObjectExpression' &&
    node.argument.properties.some(property => 
        property.key &&
        property.key.type === 'Identifier' &&
        property.key.name === 'coolKey'
    )

Putting it all together, we can now match both the function name and the return statement. We know the return node is nested within the function declaration node. Therefore we'll only match the return statement if we're already within a `coolFunction` node by setting a flag `isMatchCoolFunction`.

let isMatchCoolFunction = false;
const onEnter = node => {
    if (isCoolFunction(node)) {
        isMatchCoolFunction = true;
    }
    if (isMatchCoolFunction && isCoolReturn(node)) {
        console.log('found it 🤘, so cool!');
    }
};

const onLeave = node => {
    if(isCoolFunction(node)) {
        isMatchCoolFunction = false;
    }
};

Please note that we're matching the `coolFunction` node again when leaving the function node so that we can unset set flag. Without this step, we'd start matching cool return statements with un-coolFunctions ... Not cool at all 💅.

Some more sincere examples I know of are from using a framework like Angular where I aimed to get the component name and template path together (`coolDirective` and `template/path.html` respectively).

angularModule.directive('coolDirective', () => {
    // component logic
    return {
        template: require('template/path.html'),
    };
});

The process would be very similar to what we just went through, use a flag to for when you've entered a component declaration, then match the template string. What if there are multiple components chained off one another.

angularModule
    .directive('coolDirective1', () => {/* ... */})
    .directive('coolDirective2', () => {/* ... */})
    .directive('coolDirective3', () => {/* ... */})

The chaining complicates things as the AST structure is a little tricky, lets look at another simplified-XML style version (with emojis for visually clarity of the hierarchy):

<body>
    <🏂CallExpression-directive3>
        <🏂MemberExpression-directive3>
            <🚒CallExpression-directive2>
                <🚒MemberExpression-directive2>
                    <🧚CallExpression-directive1>
                        <🧚MemberExpression-directive1>
                        <🧚arguments-for-directive1 />
                    <🧚/CallExpression-directive1>
                <🚒/MemberExpression-directive2>
                <🚒arguments-for-directive2 />
            <🚒/CallExpression-directive2>
        <🏂/MemberExpression-directive3>
        <🏂arguments-for-directive3/>
    <🏂/CallExpression-directive3>
</body>

First thing we might notice is the reversed order as it starts with number 3; however, that's of little concern. Note that the `CallExpression` is where we'd match the component declaration, but it's within the `arguments` that we'd match the return statement and where the template path is defined. The challenge is that the `MemberExpression` is before it's sibling node "arguments", and the `MemberExpression` is what contains the next `CallExpression`. The result is that we'll hit all of the `CallExpression`s first before we hit any of the `arguments`, and what's more when we do get the `arguments`, we'll get them in the opposite order. Again 100% explicit the order we'll enter these nodes is (skipping the `MemberExpression`s:

CallExpression-directive3, CallExpression-directive2,
CallExpression-directive1, arguments-for-directive1,
arguments-for-directive2, arguments-for-directive3,

Once we know this, solving this problem becomes simple, we can keep track of what CallExpressions we've entered with a `stack` approach, and pop and push appropriately.

const isCoolChainedDirectiveDeclaration = node => {
  /*Here well match the CallExpressions that are "directives"*/
};
const getDirectiveName = node => {
  /* returns the component/directive name i.e. "coolDirective1" */
};
const isDirectiveReturnStatement = node => {
  /*Here well match the the return statements containing the template path*/
};
const getTemplatePath = node => {
  /* extracts the template path from the return node */
};

const coolDirectiveStack = [];

const onEnter = node => {
  if (isCoolChainedDirectiveDeclaration(node)) {
    const directiveName = getDirectiveName(node);
    coolDirectiveStack.unshift(directiveName);
  }
  if (
    coolDirectiveStack.length 
    && isDirectiveReturnStatement(node)
  ) {
    const templatePath = getTemplatePath(node);
    const directiveName = coolDirectiveStack[0];
    console.log(
      `Yew!🤘The directive ${directiveName}'s 
      template is at: ${templatePath}`
    );
  }
};

const onLeave = node => {
  if (isCoolChainedDirectiveDeclaration(node)) {
    coolDirectiveStack.shift();
  }
};

The approach is very similar to how we matched our coolFuntion/Return, except we're using a stack instead of a flag to keep track of how nested we are in the tree. Note as a personal preference I use .unshift and .shift, Rather than .push and .pop as I find `stack[0]` to be an elegant method for accessing the last thing on the stack, rather than `stack[stack.length - 1]`.

None of these examples will likely be of direct use to you, though the techniques of matching to multiple nodes within the broader pattern you want to match, while keeping track of where you are with the AST should be applicable for you matching advanced and deeply nested patterns within your code.

Thinking of your code as data with ASTs

Sat, 04 Jan 2020 13:00:00 GMT

ASTs (Abstract Syntax Trees) are an appealing tool for developers. If you don't know what they are, see my references below -- it's worth getting a base understanding so that you'll know when they could help you. One much spoken of possibility of ASTs is automatically refactoring code, so-called code-mods.

Another compelling application for them is to gain insights into your code or to help speed up tasks that your team needs to do every day. For example, a tool currently being worked on at Accelo is a component-to-URL tracker. What's the use case? Let's say a dev is fixing a bug, and in the process, they make a modification to the component `CoolComponent`. Given the name of this component, the tool gives a series of URLs that use the component. The URLs can be used for testing, either by the dev or passed on to our QA team.

It's worth mentioning that the tool is highly specific to our codebase as it deals with:

Perl and template-toolkit renders some HTML.
The rest is mostly angular 1.x code.
The is a broad spread of style within our angular code.
There is some jquery in dark corners.
We've recently started adopting react.
It understands our file structure.

Which mean there would be little benefit to sharing the tool.

Without the tool being able to regression test components directly in our app is either guess-work that is likely incomplete or a gruelling task of chasing all the links through files to find each use. The latter option is more robust and is what the tool does for us now. What we've done is taken developer knowledge and embedded it in the tool so that it no longer exists only in our heads. Here lies a subtle befit of such a tool, that it becomes a type of living documentation. If otherwise I was tasked with documenting the steps a dev needs to take to track a component to the URLs, I would be hesitant. This kind of documentation is likely to get forgotten about, out of date and become a source of misinformation. But embedding the knowledge into a tool ensures that it stays up-to-date so long as the tool is used. As I mentioned above, about what our tool deals with, having multiple frameworks, and so on. It means that there is a lot of dev knowledge now bundled into it.

So that's our use-case. You could do the same for your codebase too, or here are some other ideas that might be useful to your team:

A tool that counts how uses of each component, to keep track of the most used ones.
A tool that warns if there is a component that is used more than 3 or X times without sufficient tests.
A tool alerts you when a component is no longer used (depending on how advanced you wanted it, it could even delete the component and submit a pull request).
A tool that warns you if a name is used twice (you might want to avoid this for components, for example).

Please note the above is probably bias to web development (as that's what I do). The point is to think of code as a source of data and to pull out useful information and in the process, document knowledge.

ASTs exist for most modern programming languages (Javascript, Python, Go and Rust to name a few), so go forth parse your codebase. Fun fact; Perl does not have an AST (I know this because I work with Perl). If you've ever heard the joke that Perl is a write-only language, it seems that extends to parsing tools too; they can't read it. That means there are only three things in the world that understand Perl code, you for a five-minute window immediately after you wrote it, the Perl engine and probably Larry Wall.

References
A good place to start for ASTs
Accelo
Perl's lack of an AST

Ego and the computer.

Tue, 31 Dec 2019 13:00:00 GMT

In 1992 Neil Postman asked in his book Technolpoly "Will the computer raise egocentrism to the status of a virtue?". Nearly three decades on from when it was published, has time proven him right?

In Technolpoly, Postman notes that the computer as a medium emphasises private learning and individual problem solving, hence the link to ego. If we look to the tech-world and Silicone Valley, both the emphasis of the medium and the prediction hold up. There are plenty of engineers in the tech sphere at work on challenging problems, and plenty of us don't bat an eye when members of this community say things like "We're here to put a dent in the universe".

However, computers and the tech world in general have changed dramatically since Postman's comments in 1992. Depending on one's usage habits, mindlessly consuming content served up to you by the algorithms of big tech companies can hardly be interpreted as "private learning and individual problem solving". Even here, thanks to the computer, it's now socially "healthy" to broadcast one's life on social media. It seems as if Postman was right in areas outside the initial scope of his book.

Facebook and social media has to be one of the most impactful pieces of technology of the past decades, especially if we measure it by its involvement in the daily lives of ordinary people. I always feel the need to ask 'Was this inevitable?' What other technological paths did we not explore, and which are we yet to explore because social media developed when it did? Is the ego-centric nature of these platforms an extension of the computer as a medium? Or is it that it's a product of the disciples of the computer, who had already subtly embraced the intrinsic ego-centric values of their craft? Probably a bit of both.

Neil Postman died one year before Facebook launched.

MRI Volume Slice - Project Spotlight

Mon, 14 Oct 2019 13:00:00 GMT

My engineering honours thesis back in 2014 was in visual neuroscience. I worked a lot with MRI brain images as a technique called retinotopic mapping is a big tool in this field. Specifically for my thesis, I was looking for novel ways to automatically segment different brain tissue from a specific type of brain data (functional MRI data). That aside, it means I have quite an affinity towards MRI data. A typical display across many different tools that deal with brain data is a 3-window view, where each window slices the brain data across a different axis - X, Y and Z. Clicking inside one of the windows affects which slice the other two windows are showing. It's hard to describe and is easier to understand by playing around with it.

I am really proud of this little project. Partly because I'm still surprised at how responsive it is even though it's manipulating quite a large amount of data on the fly (about 20mb), but also because I built it while on holidays just a few months after I had started my first developer job. At the time, I was still most comfortable with vanilla-js and as such, there's no framework for this simple app. I think this is still mostly appropriate, especially because of its small scope and because it makes such heavy use of canvas though I am often tempted to write a React wrapper for it.

One of the most interesting challenges with this project was dealing with the 3D data and being able to grab a "slice" of the image from multiple axes. One of the things that makes it interesting is the data comes in one big flat array and I specifically chose to keep it in this format and write my own abstractions to get the data I needed for any given slice.

I also took full advantage of Hacktober-Fest (2018) and got a number of people to work on the project. While I wrote the core of the app that dealt with the data, many of the UI elements such as the loading bar, the upload field and the fetching of initial data was contributed to the project by others. For someone so new to dev work it was a great learning experience, writing up issues to be tackled by a random developer, as well as the review process for when dealing with people I didn't know but still wanting to keep a high standard of the repo. Honestly though, it wasn't hard as I had a number a great people contributing and a few that came back for more because they enjoyed it so much.

As a closing note, the data you see in the app is from OpenNeuro, a database of publically available neuroimaging data provided by researchers. This is not the only website that fosters open-science around brain data, but I think it's pretty neat that this data is available for someone like me to mess around with. (Note the face has been removed from the data set for privacy reasons).

References

OpenNeuro
The Repo
The App

React hook - lazy loading pattern

Mon, 07 Oct 2019 13:00:00 GMT

Lazy loading some data, that you also want to save into your app data store is a common enough task to warrent abstracting to a common pattern.

TLDR

Use a consitent convention for naming hooks following this pattern, for example `use<dataName>Data` so useFilterData, useActivityData etc.
Always return an object from the hook with data, loading and error properties. This will give them a predictable API.
The hook should take any arguments that are relvant to the specific data, So we might pass in the current tab to the useTabData hook.
The hook should return data from the store by default, and only make async calls if that data doesn't exist (and then update the store after fetching it).
Optional: Along side data, loading and error; a forth property onUpdate, which is a callback, can be returned for when the data also needs to be saved in the db

The goal of this pattern is to make interfacing with data from endpoints have a consistent feel as well as taking advantage of lazy loading content, and not re-fetching if the data alreay exists in your data store. Here an minimal example app I made to demo the pattern.

So how would we write a hook using this pattern, lets say the end goal is ta make a custom hook called `useTabData`

const {
  data,
  loading,
  error
} = useTabData(tab)

It will fetch data depending on what tab the user has open, following this pattern we should always return an object with data, loading and error so our component can deal with these situations. And now the logic internal to this hook is basically going to check if the data is in the store and only make a async request if it's not there. Here's an example.

import { useState, useEffect } from "react";
import { useStore } from "./store";

export const useTabData = tab => {
  const [state, dispatch] = useStore();
  const coolData = state && state.coolData && state.coolData[tab];
  const shouldFetchData = tab && !(coolData && coolData.length);

  useEffect(() => {
    if (shouldFetchData) fetchData(tab);
  }, [tab, shouldFetchData]);
  return {
    data: coolData || [],
    loading: isLoading,
    error: ""
  };
};

Basically we are getting our state from our store, and if there is no data for the current tab, we'll skip running `fetchData` inside the useEffect. Of course returning data, loading and error. Filling this in a little more by defining fetchData.

export const useTabData = tab => {
  const [isLoading, setIsLoading] = useState(true);
  const [state, dispatch] = useStore();
  const coolData = state && state.coolData && state.coolData[tab];
  const shouldFetchData = tab && !(coolData && coolData.length);

  const fetchData = async curretTab => {
    setIsLoading(true);
    const promise = new Promise(resolve => {
      setTimeout(
        () => resolve(tab === "sweet"
          ? ["a", "b", "c", "d"]
          : ["x", "y", "z"]),
        3000
      );
    });
    const data = await promise;
    dispatch({
      type: "UPDATE_COOL_DATA",
      payload: {
        type: currentTab,
        data
      }
    });
    setIsLoading(false);
  };

  useEffect(() => {
    if (shouldFetchData) fetchData(tab);
  }, [tab, shouldFetchData]);
  return {/* data, loading, error etc*/};
};

So were using another bit of state at the top of our hook for isLoading, setting that to true at the stare of a call to fetchData, then we make our async call (mocked here with a timeout), once the data comes back we'll update the store and set isLoading to false. All is well with our frondend 🤗.

The above example can be seen here, Or codesandbox (styles arn't working in the sandbox 🤷‍♀️).
The Repo can be found here, (note, that it's writen in typescript).
Here's a blog post detailing a react hooks data-store similar to the one that I'm using in the example. (though the type of store doesn't matter)

Some other points

Note that in the example the data being returned is coming directly from the store (assigned to a varible inbetween). It might be tempting to put the data from your async call into a local useState, but I would highly discourage it as duplicating state like this could lead to some nasty bugs.
Also adding a onUpdate callback, for data that can be save to the backend from the fronend. This callback would obviously be making async calls and so It also should return a similar format of data, isLoading, error (maybe status instead of data, if you're not interested in the data comming back).
We would implement the returned error in a real example.
You might get stale data if you only ever return from the store. For most data this isn't likely to be a huge problem, but some extra logic to invalidate/timeout old data could be implemented.
If you find you're writing a lot of hook like this for you app, you migh want to abstract some of this into a hook factory in order to reduce duplication (i.e. the code that checks if the data is already in the store). I might come back and write more details about this. 🤞

Use Apollo with Gatsby (or SSR)

Sat, 05 Oct 2019 14:00:00 GMT

So the short of it is, you need to use `isomorphic-fetch`.

There's no point me repeating what's already been said well in the apollo-docs, so start there for setting up apollo-client. However you might find everything is working in dev mode, but your project won't build 😔.

Here's the fix, first:

npm install isomorphic-fetch
# OR
yarn add isomorphic-fetch

Then, where you have initialised your apollo client you need to:

import ApolloClient from 'apollo-boost';
import fetch from 'isomorphic-fetch'

const client = new ApolloClient({
  fetch,
  uri: 'https://48p1r2roz4.sse.codesandbox.io',
});

Why? It's because the ApolloClient by default uses the browser fetch api, and because Gatsby statically builds your website in node (and fetch doesn't exist in node). Luckily the ApolloClient configeration options allow us to pass in a fetch compatiable api, of which isomorphic-fetch fits the bill nicely 👌.

Reference:

Apollo react docs again
isomorphic-fetch
Github issue that help me solve this initially