Building an internal documentation website in three days using Contentful
How to use the fail-fast philosophy to find the right tools and setup to build a small, completely free and fast internal documentation website for your team.
Last week I took it upon myself to try and find out a suitable online documentation tool for our small team of three developers. Our code base had split into four repositories already and I wanted to make everyone’s work a little bit easier. Everyone worked on separate repositories, but needed information on all of them.
There were a few requirements:
- The tool had to be SaaS-service or serverless (no on-premise installations)
- It had to be either free or minimal cost
- It had to allow the creation of static pages, perhaps upload images and contain a search
- It needed to contain some type of user authentication as content is supposed to be internal
I discovered a few open source solutions, but nearly all of them were on-premise installations. So, I ended up with a few options:
- We could host documentation on Github with markdown files, but that would mean we’d have files on every repository separately making it hard to read.
- We could use Gitbook, but it costs approximately 400 € per year for a team of minimum 5 users. It’s free for personal projects (1 user) so free tier wasn’t an option.
- I could try and find some platform that allows free tier usage and program the app myself.
Github proved to be too complicated (documentation split between multiple repos) and Gitbook too expensive for our taste, so I decided to make this a mini-project to which I allocated three days.
Three days and five steps.
- Define
- Plan
- Design
- Develop
- Deploy.
Step 2: Plan
As I had already defined what I needed, I had to move to planning. I started out by creating a list of required features.
- User authentication
- Static pages with CRUD(I) actions
- Navigation based on pages and hierarchy
- Page groups, a way to set up subheadings in the navigation
- Images and attachments
- And as a bonus: Search functionality.
After this I took on a fail-fast mentality of trying out a few options for the stack.
Option 1: Cosmos DB and Azure functions with API Management for the back-end
We use Azure as our cloud platform to host our services. I knew the front-end would not be a problem if I could find the optimal solution for the back-end. I went and searched for an Azure equivalent of AWS DynamoDB, which is a NoSQL database that you can practically run for free with our level of use. We’re not using AWS so DynamoDB was not an option, though.
Turns out, Azure had such a database called Cosmos DB. The internal data model is exposed through a proprietary SQL API or five different compatibility API’s, one of which is MongoDB. I took a quick glance at the pricing and didn’t really understand what 100 RU/s meant at first, but we’ll circle back to this.
I had already selected the language for Functions, which was gonna be JavaScript (Node.js) as it’s most familiar to me right after PHP. The set up was pretty simple. Azure has a nice tutorial from which I devised a database module and added simple write+read helper functions.
I set up a development project and tested the connection. All fine, so far. After this I migrated the code to Serverless framework for easy deployment and tried the API. All went fine so far and I could add, update, delete and list items from the collections.
Then I took a look at the cost management. The bill was already 0.30 $ in a few hours. I am sure I set up the database provisioned throughput wrong, as Cosmos DB is billed by it and in West Europe this cost is:
100 RU/s single-region account 1 x €0.0068/hour.
In comparison DynamoDB pricing consists of separation between write and read, write being more expensive.
Write request units $1.4135 per million write request units
Read request units $0.283 per million read request units
In DynamoDB a write request unit is up to 1 Kb and read up to 4 Kb. In Azure I believe both are 1 Kb.
Small tip: As the pricing suggests, you are best to avoid full scans in DynamoDB, so plan your partition and sort key well! That way you can perform smart queries and avoid cranking up your bill by poor design. I’ve used DynamoDB for logging user actions in a web application, and performing a search for a specific partition key containing the user ID and with a specific time range (partition key) you only search what you need, nothing more, saving you a ton of money.
Now based on this I finally figured out Cosmos DB is billed like any other database type with cloud providers, based on the time your database is online. The hourly rate depends on our configured RU/s settings.
DynamoDB however is billed based on the request units you consume. If you can stay under a million request units for write and read, you might survive with a zero dollar bill.
So, I abandoned Cosmos DB and Azure quickly as an option. This was a failure.
In retrospect we would have survived with a $4.8 monthly bill by setting the database down to 100 RU/s per hour. However, I only discovered this later as I went through the documentation to understand how the pricing worked.
Option 2: Contentful, Gatsby and Netflify
I never used Contentful before, but I had heard of it. It’s basically a headless CMS. You can design your content models, and thus entries, that are exposed via a REST API, or a GraphQL Content API.
The price was free for a community plan of maximum 5 developers. It allows 2 M requests and 25 000 records. Plenty for our documentation website with proper planning.
Contentful offers you a straightforward tutorial when you register for the platform, with Gatsby (React framework) and Netlify (deployment platform). I took a quick look at the example project and it seemed to work fine, so I continued with Contentful.
Throwing Gatsby out the door
When attempting to work with Gatsby I came across three separate errors from the start. I can’t recall these anymore. I managed to quickly fix them or avoid them, but the thing that caused me to abandon the framework was the fact that Contentful served the user access token as a hash URI fragment:
page#access_token=<ACCESS_TOKEN>&token_type=Bearer
Gatsby’s gatsby-react-router-scroll package interpreted this as an anchor on the page and attempted to scroll to it, failing and crashing when the ID did not exist. I tried to overwrite this functionality to no avail and even created an issue on their Github repo.
I didn’t want to waste days on this issue, so I considered it a fail and moved to another framework.
Hello, Next.js
I never worked with Next.js before. I never worked with Gatsby either. I didn’t even know Next.js was developed by Vercel, where I’d just recently deployed another project for a customer. All I knew was it was a very popular React framework so I gave it a try. That meant Netlify was out too as I was pretty sure deploying to Vercel was easier.
The first things I tied was routing. Then the user authentication where I’d just been halted. Both worked fine. Then I tested the API calls to Contentful and those worked out well too, so there was no showstoppers.
I had already deployed a previous project to Vercel and Next.js was their framework, so I knew it’d work. Time to move on with Contentful, Next.js and Vercel.
Step 3: Design
This was probably the fastest step I took. I didn’t want to spend too much time on the design. I simply decided it had to be simple and readable. I also didn’t want to concern myself with mobile support for now, even though I usually develop websites mobile first.
I browsed a few shots from Dribbble for reference and went with a simple three-column solution that resembles our public documentation site, created with MkDocs. I had created our design system using atomic design, so I used existing components to create a simple layout.
By now I’d spent the first day building proof of concepts, testing and failing fast. After I figured out what worked I completed the design quickly and on the second day I could move onto development.
Step 4: Development
Next.js offers the same basic tools as any React framework.
- Easy to set up
- Static exporting
- Pre-rendering
- Optimized production build
- Routing
- TypeScript support
It helps you by handling the boring stuff like writing your own Webpack configurations, bundling or routing. You can just install the framework and start coding pages.
Redux for state management
I had created a content model inside Contentful, called Page, which contained a title, content, author information, creation timestamp, update timestamp, order number. I fetched all of these entries to render the sidebar navigation based on the order. I later added grouping based on Page group, a second content model.
I soon realized that by storing all the pages inside a state container would allow me direct access to each entry making it possible to create an editing form using the already stored data, and not fetch it again when the form page loaded. This would reduce the number of API requests and since Contentful’s entry update requires you to send in the entire entry, as all previous data will be erased, this proved to be quite useful.
I used Redux toolkit, the official and opinionated toolset for Redux which makes using it super simple. You need to install the package, then set up a store file.
import { configureStore } from '@reduxjs/toolkit'
import pagesReducer from './reducers/pagesReducer'
const store = configureStore({
reducer: {
pages: pagesReducer,
}
});
export default store;A simple reducer.
import { createSlice } from '@reduxjs/toolkit';
const pagesSlice = createSlice({
name: 'pages',
initialState: {
pages: []
},
reducers: {
addPages (state, action) {
state.pages = action.payload;
},
addPage (state, action) {
state.pages.push(action.payload);
},
updatePage (state, action) {
const pageIndex = state.pages.findIndex(page => page.sys.id === action.payload.sys.id)
if(pageIndex > -1) {
state.pages[pageIndex] = action.payload;
}
}
}
});
export const { addPages, addPage, updatePage } = pagesSlice.actions;
export default pagesSlice.reducer;(Redux Toolkit uses Immer to handle immutability while making it really simple to update your state.) After this you need to provide the store inside the Next.js _app.js file.
import { Provider } from 'react-redux';
import store from '../src/store/rootReducer';// ...
const App = ({ Component, pageProps }) => {
return (
<Provider store={store}>
<Component {...pageProps} />
</Provider>
);
};
You now have access to your state from any component with the useSelector hook.
const pageState = useSelector(state => state.pages);Now that I’d set up Redux I could navigate between documentation pages without having to make additional API requests. The state already had all the pages in it.
Of course this means the initial collection request might be heavy, depending on the size of your pages. Then I would recommend switching tactics and only fetching the titles and slugs for the entries on the sidebar, and requesting page contents on each page load.
But, whatever works best for you.
Contentful’s Content Management API
The documentation for the Content Management API is pretty good. The website contains good navigation and examples for multiple solutions. Contentful provides SDK’s for multiple languages to make it easier to work with the API. All I needed was the User and Entries.
To gain access to the content you either need an application level access token (public), or user specific access tokens. As I wanted the website to be limited to certain people, I selected user access tokens that were provided by Contentful’s OAuth authentication flow.
I used the useEffect hook to fetch data once the component is mounted. I would prefer to use pre-rendering, but I could not gain access to the access token in the cookie from any server-side or static render method.
useEffect(() => {
const client = createClient({
accessToken: Cookies.get('token')
});
client.getSpace(process.env.CONTENTFUL_SPACE_ID)
.then((space) => space.getEntries({
content_type: 'page',
order: 'fields.order'
}))
.then((response) => {
dispatch(addPages(response.items));
})
.catch((e) => setError(e));
}, []);As you can see I store all received entries and I can access them from the container to load up any page immediately without additional network requests making the app really fast.
Pitfalls to this approach
Like I said earlier, there are some pitfalls to this.
- If your pages get too big you might have huge network requests for initial load
- If you have multiple pages (over 100), you need to set up pagination as Contentful limits entries by default to 100, but this could provide useful if you implement lazy loading on the sidebar.
Our pages only contain simple HTML and they should be relatively small, so this works for us for now.
Final result
I quickly went through the rest of the pages and functionality in 1.5 days. Once the user gained an access token, he was redirected to a page that listed the documentation pages on the sidebar, and page contents on the right.
Pages can be added, edited or deleted. All pages can be assigned to a parent, creating nested navigation items. They can also be added to specific Page Groups creating subheadings. Navigation is printed in order.
I’ll explain some features in detail below.
Implementing Search with Contentful
I used only 30 minutes to implement the search functionality as Contentful contained a full-text search endpoint, which proved to be quite handy for a website of this type. Pages can be searched using any type of keyword and the results are listed below the search (see image above).
Below is a sloppy example on how this works. Basically any keyword that is at least four characters long triggers a search after 0.5 seconds. The results are listed below the input and you can select the page you need.
const handleSearchInputChange = (e) => {
e.preventDefault();
e.persist();
let typingTimeout;
setOpen(false);
clearTimeout(typingTimeout);
const value = e.target.value;
const client = createClient({ accessToken: access_token });
if (value !== '' && value.length > 3) {
typingTimeout = setTimeout(function () {
client.getSpace(process.env.CONTENTFUL_SPACE_ID)
.then((space) => space.getEntries({
content_type: 'page',
query: e.target.value
}))
.then((response) => setResults(response.items))
.catch(console.error)
.finally(() => setOpen(true));
}, 500);
}
};return (
<Search ref={ref}>
<input type="text" placeholder="Search..." onChange={handleSearchInputChange} onFocus={() => results ? setOpen(true) : null}/>
{!isEmpty(results) ? (
<SearchResults>
{results.map((entry: Entry) => {
let content = get(entry, 'fields.content[\'en-US\']', '');
if (content) {
content = htmlToText.fromString(content).substr(0, 64);
}
return open ? (
<SearchResult>
<Link href={'/page/[id]'} as={`/page/${entry.fields.slug['en-US']}`}>
<a onClick={() => setOpen(false)}>
<h4>{get(entry, 'fields.title[\'en-US\']', '')}</h4>
<p>{content}...</p>
</a>
</Link>
</SearchResult>
) : <></>;
})}
</SearchResults>
) : ('')}
</Search>
)
Implementing image upload to Contentful with React Draft Wysiwyg
I used React Draft WYSIWYG as the content editor, because it came with the required editor functionality and I didn’t have to use the framework to build them myself. I did some fine tuning with the styles and found a nice Github issue post about how to implement a file upload.
Basically the Editor required an uploadCallback that returns a promise.
<Editor
editorState={editorState}
toolbarClassName="wysiwyg-editor-toolbar"
wrapperClassName="wrapperClassName"
editorClassName="wysiwyg-editor"
onEditorStateChange={onEditorStateChange}
defaultContentState={get(entry, 'fields.content[\'en-US\']', '')}
toolbar={{
image: {
uploadCallback: handleImageUpload,
previewImage: true,
alt: { present: true, mandatory: false },
inputAccept: 'image/gif,image/jpeg,image/jpg,image/png,image/svg'
},
options: ['blockType', 'fontSize', 'inline', 'list', 'textAlign', 'link', 'image'],
inline: {
inDropdown: false,
options: ['bold', 'italic', 'underline', 'strikethrough']
},
link: { inDropdown: false }
}}
/>The upload function.
const handleImageUpload = (file) => {
return new Promise(
(resolve, reject) => {
const client = createClient({
accessToken: Cookies.get('token')
});
client.getSpace(props.spaceId)
.then((space) => space.createAssetFromFiles({
fields: {
title: {
'en-US': file.name,
},
description: {
'en-US': file.name,
},
file: {
'en-US': {
contentType: file.type,
fileName: file.name,
file: file
}
}
}
}))
.then((asset) => asset.processForAllLocales())
.then((asset) => asset.publish())
.then((entry) => resolve({ data: { link: entry.fields.file['en-US'].url }}))
.catch((error => reject(error));
}
);
};This function returns the image straight into the editor. What we’re doing here is taking the File provided by the editor’s upload dialog, and sending it to Contentful to create an asset. We then receive the asset, so we process and publish it. Finally we receive the Entry that contains the URL that we can resolve and return to the editor.
Deployment
Deploying a Next.js app to Vercel is a walk in the park. The documentation is pretty straightforward: Just link your Github repo to the app and let it do the work. Worked out like a charm and the app was soon online and worked perfectly.
What now?
A day later I realized Atlassian’s Confluence had a free tier for a team of maximum 10 people. I hadn’t started writing content yet, so I considered abandoning the project right after it was done to use Confluence.
But Confluence is slow. It’s slow to load and slow to use. I have a lot of prior experience with it and I hate it for this fact alone. It’s a great tool for enterprises, but not for quick software documentation with small teams.
I think I’ll stick to what I created and as soon as I’ve cleaned up the code, release it as an open source repo if anyone wants to take a look. I might even continue developing it, but my time is rather limited with other work as well. This was a short project during my summer holiday.
I can totally recommend Contentful as a CMS for your front-end solutions. You might have to contact their support to ask for an enterprise deal if you’re thinking about developing a site or an app with heavy network traffic.
