The semiconductor shortage led to one of my most successful projects: ilox, a product stock tracking bot. It monitors changes in product stock and sends out notifications whenever it changes.

This guide will show you how to write a monitoring bot yourself using NodeJS and cover the following topics:

  • setting up a scalable architecture for a tracking bot
  • different crawling methods: source code parsing, using APIs and automated browsers with Puppeteer
  • advanced tips (avoiding bot detection, logging, proxies and more)

Keep in mind that this is not a guide on how to crawl a specific shop - we're going to use a fake "shop" for this which can be found here.

As with all my guides, the entire source code will be downloadable at the end of the article.