April 8, 2021

If you have wondered this at any point, you are right where you are supposed to be !

In my last article, I already tried to help these pain points by showing you how to deploy your own custom API in a few minutes.

Today, we are going to talk about unoffcial APIs, a way to alleviate even more frustration of not being able to automate a process on a given SaaS.

Before we get into it, let's talk about a tangible example : Notion.

Notion has become one of the top, if not the top, workspace/organizer app in the span of a few months. Its design is top notch and endlessy customizable.

Now that you're reading this article, you probably know that it has also a major flaw. (aside from being slow at times)

It has no official API, so there is no official integration, no Zapier, nothing !

That is until you discover an unofficial API.

An unofficial API is basically a reverse engineering of the official API using the frontend of a service.

If you manage to intercept the calls made when you add, update or delete an entry on Notion's interface, you can reproduce them with any programming language. In essence, it's a technique that is very close to scraping.

For common services, there are community made unofficial APIs like this one easily found on GitHub.

As mentioned on the repository, the creator of this unofficial API uses Notion as a CMS and his API to download the notion pages. 

He then uses a flow to convert the downloaded data to html and then publish his new page automatically.

This means we can start building cool stuff like :

As another example, we recently made the switch to webflow at Quable, after the OVH burnt server fiasco.

We hosted Quable's homepage on Wordpress hosted on a burnt OVH server, and we had no backup !

We had to react fast, in particular not to lose our SEO efforts...

We had over 300 blog posts, in French and in English to redirect somewhere and not to a dead 404 page !

The plan was to set up an emergency website on Webflow, and redirect as much links

The problem is : Webflow has an API, but they do not do bulk redirections, so the only solution would be to add every single redirection by hand ...

Webflow redirect
Webflow redirection UI

That is unless you know about a little trick related to unofficial APIs.

The idea here is to automate the process of:

  1. Finding the 300+ links that need to be redirected somewhere (because remember: we didn't have website to even look links up)
  2. Adding them to webflow in bulk without doing it by hand

For the 1st point :

I used APIfy to scrap the first 300 Google links returned by the query `site:quable.com` which hopefully gave me the most important links of the website.

apify crop
ROUGH crop APIfy Google Scraper

Once I had my links, I needed to automate my redirections, and I did so unofficialy, with the help of two things : the F12 key and this cURL syntax converter tool.

To add multiple redirections, I had to reverse engineer how my browser actually sends a request to Webflow's server to add a redirection and then write a script to execute the requests over my scraped links.

To know how my browser handles the request, I use the F12 key which shows the Developer Tools on any common browser, and checkout the Network tab.

Developer Tools Crop

To watch the request, I clear the console by clicking on the "forbidden" icon and I add a random redirection like /testA to /testB :

Requests from Developer Tools

Here it's pretty easy : the route is explicit "redirect".

To replicate it, I right clicked on it, clicked on "copy as cURL" and used the cURL converter tool to translate it easily to Python:

Copy as Curl crop

(Careful when sharing this kind of information especially here where there are cookies containing login data !)

When pasting, the Python code appeared instantly, and in there you can clearly see the our "testA" and "testB" redirection paths !

The rest is simple as I just needed to make a function out of this thing and loop through with my scraped links:

And that's how I automated the whole process creating a mini unofficial webflow API to suit my needs !

Nowthat I have explained the magic, this type of reverse engineering comes with a few warnings:

In this specific case, I put random timeouts between each added redirection between 10 and 100 seconds, and everything went well !

Let me know if there are specific services you would like to see automated this way !

Until next time,


Get my posts in your inbox

I promise I'll never send any spam your way

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.