Modern HubSpot CMS Development Flow With CI and GIT

Updated Nov 13, 2024 • 14 min read

HubSpot is a great platform for marketers and content creators. It gives them the power and freedom to create the pages they want easily, without the need to involve developers in the process.

I think it is really important for content creators to have that freedom – if we expect them to deliver high-quality content, we have to give them the right tools.

On the other hand, from a developer's perspective, you can focus on creating components instead of formatting content and pasting that brand new article that your content creator provided in a DOC file.

Using HubSpot forces the development process to take place in the browser (via the HubSpot admin panel). You have to create components and templates using plain HTML, CSS, and JS. Usually, a change means saving and refreshing the page to verify whether it works as expected. Development and testing usually happen in a production environment and you have to be careful not to break anything. There is no chance for rollback or automated backup (you can always paste components’ source code to your notepad though). To make matters worse documentation is quite laconic.

The problem

While HubSpot's default workflow may be good enough for Auntie's blog with cute cat pictures, it is hard to apply that flow in a huge team with high quality expectations. Developers will start to accidentally overwrite each other’s work, there will be no control over changes, nor a chance to test everything properly. Smells like a catastrophe in the making!

The time when front-end developers were using plain HTML and CSS to build webpages and upload them with FTP is gone. Nowadays, we are used to those fancy dev tools, transpilers, task runners, the code review process, development flows with continuous integration, and automated tests. And we use them for a reason – to make our work easier, more efficient, and ensure better quality solutions.

But is there a way to use modern development techniques with HubSpot? Luckily, after a lot of research, experiments, and out-of-the-box thinking, I might have found a solution (or at least an improvement).

Goal

I decided to start with a proof of concept which will show what we can do with HubSpot, so I focused only on page templates, custom modules (components), and static assets (like icons). I believe that’s enough to create even a complex website, but there is a chance that you may need more, or that it won't fit your case.

So my plan was to:

Have a local development environment with live preview.
Have the ability to develop custom modules and templates locally.
Track changes and do code review in Git.
Use the build process with SCSS support and possibility of transpilation.
Build and deploy the whole app with a continuous integration tool.

In other words, I had to find a way to create modules outside the HubSpot panel, run them locally and then somehow push them to the HubSpot environment. Sound easy?

If you are not interested in a story about how I came up with the solution, you can skip directly to it.

Process

I started by digging into HubSpot's documentation. I focused on modules, templates, and assets – how they are structured, what the API is, and flexibility. Then I looked at how I can integrate with HubSpot – what are the available APIs and ways to exchange data. I was surprised that the information about local development was vague or not present at all. Although I found that you can upload modules using an FTP account, it didn’t specify how to create them and what the API is.

Then I started researching online – mostly HubSpot forums and Stack Overflow. Accidentally, I spotted THIS repo which was a breakthrough for this story. It is a dockerized local-hubl-server which seems to provide a HubL interpreter with a local HubSpot components API, and (more importantly) support for modules. Unfortunately, it is poorly documented and it's hard to find any mention of it in HubSpot’s documentation (even though it’s an official package from this company).

So I started wondering – how do you create a custom module? You can use the web panel but I wanted developers to create components locally and push them to the web panel after code review. While going through my HubSpot website FTP, I noticed that custom modules are available there, with configuration and dynamic fields defined in JSON files – that's good news! Unfortunately though, when I tried to upload a directory with a new module, it was not shown in the web panel.

So the search for a custom modules API continues…Luckily, I came across official documentation which seems to specify custom module fields pretty well. So I created meta, fields, and source files (HTML/JS/CSS), uploaded with FTP and the custom module was shown in the web panel. Great success!

Now that I had a way to run local-hubl-server and knowledge on how to create new custom modules locally, I could focus on creating a simple development environment to meet the goals I specified in the beginning.

I started playing with local HubSpot CLI and it seemed that it could even fetch data from my HubSpot panel, although there were a lot of unknowns on how to use that data (again – laconic or no documentation). So I decided to leave it as is and focus on creating modules and templates only (with mock data), as it is good enough for development (there might be a problem if you want to, for example, fetch data from HubDB or use some specific HubL functions, but still, it is only a proof of concept).

At first, my local modules didn't work in the local server. After a lot of time wasted on debugging, I found two things – the component used locally needs to have a unique ID set in meta.json, and to include a custom module created that way, you should use the

{‎%- module "module_name" path="/Custom/path_to_module_dir" -%‎}

tag with an absolute path to your custom module.

At this point, it seemed that I had everything I needed. The only thing left was to prepare a build process for the modules and see if that worked. I decided to go with Gulp as it seemed like a better fit than Webpack (as it is a task runner and I wanted to run tasks, not bundle JavaScript). So after a bit of work on Gulp tasks, paths, and a watcher, I finally came up with a solution.

Solution

You can see my proof of concept here: https://github.com/macap/hubspot-workflow – it contains a README file that should be good enough to test its capabilities, but I'll also try to describe it briefly here.

To use it, some experience with HubSpot is required, especially:

How custom modules work and how to create them.
Commonly used template tags.
HubSpot’s directory structure.

It's worth noting that the hs-cms-server requires Docker to be running on your machine.

Overview

Looking at the HubSpot workflow repo you'll notice:

.circleci dir which contains CI configuration (obviously) and deploy scripts.

dev directory is where `@hubspot/local-cms-server-cli` lives – it will be our development environment, which will run modules and templates locally.

src is the most important place – this is where you will put all components source code. It is divided into assets, modules, SCSS (SCSS libraries like bootstrap), and templates. There is also a global.scss file.

templates – This is where I kept boilerplates for module and template which Gulp uses to generate a new component.

Main package.json which contains packages used by Gulp.

Gulpfile.js which defines Gulp tasks as building CSS from SCSS, altering modules JSON config to make them work with a development server, watching changes and copying src files to local-hubl-server.
.env.example which is a template to create your .env file with configuration (only PATH_PREFIX so far).

The basic flow is that after setup (described in README), you can create modules and templates with Gulp command and then if the dev server and Gulp watcher are running, those components are built on every file change so you can view them in the browser. When you're done with development, you push your changes (using PR) to the remote Git repository, and after review and merge to master, the build process starts. On build, CircleCI will fetch your repo, build all components and assets for production, and upload them to the proper directory. And it’s done!

You just have to agree with the team that the Git repository is your source of truth – so any changes in components’ source code via HubSpot web panel can be lost. But that's a good thing!

Context

I did not research local-cms-server-cli in detail, but I noted a few things. dev/context directory contains all "mock" data which is used by the local HubSpot server. If you initialized hs-cms-server with --context flag, you will find some default content for the portal with ID 123. There are example pages, blog posts, and menus.

If you want to alter the example page template, go to dev/context/content/123/, find the JSON file representing the page – 6999365956.json in my case and the template URL for page template_path. I updated the path with "Custom/v3/templates/main.html" in my example.

A similar case is for blogs – in dev/context/blogs/123 you will find blog data, and you can update “Item_template_path" and "listing_template_path" keys to fit your needs.

If you want to render a menu, you will find its data (and most importantly, its ID) in dev/context/menus directory.

There is also a cli-config.yaml file in the dev directory. You can set portalId for the local server there. There is a feature that allows you to download content from your HubSpot instance and use it locally, but I haven't tested that yet.

Templates

If you want to keep all code in the Git repository, there is no way to use dynamic templates, so you have to stick with classic ones. HubSpot requires templates to contain

{‎{ standard‎_header_includes }‎}

and

{‎{ standard‎_footer_includes }‎}

you won’t be able to upload them if those tags are not present.

To include a module, you can use

{‎%- module "myModule" path="/Custom/v3/modules/myModule" -%‎}

if your module is in src/modules/myModule.module path. You can also add a CSS file using

{‎{ require_css("/assets/global.css") }‎}

if your file is in src/global.scss.

You can generate a new template from boilerplate using gulp templates.new --name template-name command.

Modules

Modules are kept in src/modules in <module_name>.module directories. In fields.json you can define the dynamic fields you’ll use, following https://github.com/bradhave94/HubSpot/wiki/Custom-Modules-JSON. Nothing has to be changed in the meta.json file for now. The HTML, JS, and SCSS files are self-explanatory and you work with them as you'd do in web panel.

You can generate a new template from boilerplate using gulp modules.new --name module-name command.

Build and deploy process

The build process is executed by simply running npm run gulp build and executing the FTP upload script written in JavaScript. To automate the process, you'll need CircleCI with the configuration described in README.

The good thing is that the script will automatically update paths to hubfs for assets.

Example

You can see a site example here and its source code using my tool hubspot-website-example. Modules and templates are created locally and uploaded after merging to master (yay!).

In the video above, you can see an example of how I update a module, review it locally, and then deploy to HubSpot using just git push.

Summary

So that's a brief description of my adventures with HubSpot’s modern development flow. This is just a proof of concept, so it may have some errors, quirks, and some features that are not yet supported (especially some of the HubL tags). To use it in production and to fit your needs it might need some adjustment, but I think it’s a good place to start and gives hope that you don't have to work in the browser to use HubSpot.

I did not create a package for this solution as I wanted to create a template that you have to adjust yourself (probably your flow will be slightly different anyway).

Known issues are:

For some reason, sometimes local-hubl-server freezes and needs to be restarted.
If you have a lot of files, FTP transfer is not reliable – the script should be fixed. It also won't delete old files, only overwrite.
To make the blog template work, I had to create a blog template in HubSpot panel, and then overwrite it with my local version. By default, new HTML templates are set as page (not blog) templates.

I believe that if you structure your code properly and don’t overuse functions provided by HubSpot, it will make your code more flexible, especially when it comes to migrating your site to a different provider. That’s a huge advantage!

What next? I think there are some bugs to fix, cases to handle, so I'd focus on that. In a local HubSpot environment, there is an option to download your HubSpot site content to the local machine and test it there, so there is a good chance that with a little effort you could use it while developing (populate module with data from your HubDB). I haven't tried that yet though. If I had more time I'd also like to add JS and stylesheet linters. I also had an idea to replace local-hubl-server with a simple HubL parser, as the original HubL server is in my experience quite unreliable. But writing a HubL compiler is a topic for another article (or series).