Skip to content
← All work

05 · Data engineering · Quant + ML

GhostCheck SA

How long do South African job listings really stay open? A daily scraper over ten employers' career sites, POPIA-safe by design, running at R0.

Code on GitHub
Employers, 5 Workday + 5 SuccessFactors
10
Scrapes returned ok
336/340
Days of daily runs
34
Infrastructure cost
R0

Open listings per employer, daily

From history/runlog.json. Gaps are failed scrapes. Click an employer to isolate it.

Problem

Job seekers in South Africa apply to listings that may have been open for months, or reposted under a new ID. Nobody publishes how long postings stay up. I wanted a dataset that answers that, without collecting anything personal.

Approach

  1. 01A GitHub Actions cron scrapes Absa, Discovery, FirstRand, Gold Fields, Nedbank, Old Mutual, Pick n Pay, Sasol, Telkom and Tiger Brands every day and commits the result.
  2. 02Each response is classified ok, blocked (403, Akamai challenge, CAPTCHA) or error. That is the main risk to the project, so it is measured first.
  3. 03A hard field whitelist in store.py keeps only job ID, title, location, URL, first_seen and last_seen. No names, contacts, descriptions or salaries.
  4. 04A locked v1 data contract with JSON Schemas and a validator, so a future static site can build against it.

Results

Between 27 September and 30 October 2026 the run log holds 340 employer scrapes. 336 came back ok. The 4 errors all fell on 17 October.

Old Mutual alone listed between 449 and 608 open roles on a given day. Telkom lists under 10.

Next

M1 in the README: closure detection, repost chains, time-to-close metrics per employer, and the public Astro site.