ThailandDev: Zero-Rule AI Scraping

See how Gemini 2.5 Flash powers a zero-rule job board scraper analyzing career pages for structured data, covering the full stack from crawling to deployment.

Gemini Crawl4AI Pydantic Supabase Next

Overview

ThailandDev is a job board for tech roles in Thailand, powered by an AI pipeline that requires zero manual scraping rules. I’ll walk through how we use Gemini 2.5 Flash to analyze any career page structure, extract job URLs, and pull structured data including skills, seniority level, remote policy, and language requirements. The talk covers the full stack: Crawl4AI for browser automation, Pydantic for schema validation, Supabase for storage, GCP Cloud Run for scheduling, and Next.js for the frontend. Adding a new company to scrape? Just drop in a URL.

Links

https://thailand-dev.com
Curated job board for international software and data engineers based in Bangkok.

Tech stack

Gemini

Google's natively multimodal AI model: understands and operates across text, code, audio, image, and video.

Gemini is Google's most capable and general AI model, engineered from the ground up to be natively multimodal: it seamlessly understands and combines information across text, code, audio, image, and video inputs. The technology is optimized for flexibility, running efficiently on everything from data centers to mobile devices. It is deployed in three key sizes: Ultra (for highly complex tasks), Pro (for broad scaling), and Nano (for efficient on-device tasks). Developers access this power via the Gemini API to build next-generation applications.

https://deepmind.google/technologies/gemini/

View projects
Crawl4AI

Crawl4AI is the open-source, LLM-friendly web crawler engineered for blazing-fast, structured data extraction and direct ingestion into AI pipelines.

Crawl4AI is an open-source, Python-based framework specifically designed for AI workflows (RAG, autonomous agents). It processes complex web content to generate clean, structured data: Markdown and JSON. The system employs intelligent, adaptive crawling algorithms to determine when sufficient information is gathered, preventing unnecessary page fetches. It offers flexible extraction strategies (CSS, XPath, LLM-based) and delivers up to 6x faster performance than traditional scrapers, ensuring high-quality, AI-ready datasets.

https://crawl4ai.com

View projects
Pydantic

Pydantic is Python's most-used data validation library: it enforces data schemas using standard type hints and boasts a Rust-core for exceptional speed.

Pydantic is the premier data validation and parsing library for Python. It mandates data structure using pure, canonical Python type annotations, drastically reducing boilerplate code. With over 360M monthly downloads, Pydantic is battle-tested: all FAANG companies and major frameworks (FastAPI, SQLModel, LangChain) rely on it for robust data handling. Its core validation logic is written in Rust, ensuring high performance. Pydantic models also generate JSON Schema, facilitating seamless integration and documentation for API development.

https://pydantic.dev

View projects
Supabase

The open-source PostgreSQL development platform: a Firebase alternative for rapid backend deployment.

Supabase is the Postgres development platform, providing a complete, open-source backend-as-a-service solution. It packages enterprise-grade tools like a full PostgreSQL database, a RESTful API (via PostgREST), and a GraphQL API (via pg_graphql) that are auto-generated from your schema. The platform includes a comprehensive suite of services: Auth (for user sign-ups and SSO), Storage (for files with S3 integration), Realtime (for database change subscriptions), and Edge Functions (for serverless logic). The core value proposition is clear: build fast with a powerful, scalable SQL database that offers 100% portability and robust security features like Row Level Security (RLS).

https://supabase.com

View projects
Next

Next.js is the full-stack React framework: it delivers high-performance web applications via hybrid rendering and powerful, Rust-based tooling.

This is the React Framework for production: Next.js enables you to build full-stack web applications with zero configuration and maximum efficiency. It supports a hybrid rendering approach (Server-Side Rendering, Static Site Generation, and Incremental Static Regeneration) for optimal speed and SEO performance. Key features include React Server Components, Server Actions for running server code directly, and the App Router for advanced routing and nested layouts. Developed by Vercel, it leverages Rust-based tools like Turbopack and the Speedy Web Compiler for the fastest possible builds and a superior developer experience.

https://nextjs.org/

View projects

Related projects

Building Faster and Closing the Loop with PostHog AI

Chicago

This demo covers PostHog's AI product and LLM observability, showing how these tools streamline shipping and provide builders…

JobsYo: Building an AI-based but Human-driven Job Search, Research and Apply Ecosystem

Toronto

See a live demo of an AI job search platform featuring multi-model API routing, context engineering, agentic job…

AI speed dating

Seattle

Learn how an AI host uses QR‑linked WhatsApp, function calls, Airtable matching, and RAG to register participants, create…

AI in Education

Pune

The talk explains Sahay, an AI platform using RAG and LLMs to deliver personalized learning paths, career counseling,…

OpenAI Releases

Chicago

This talk provides a clear overview of the new developer tools released by OpenAI, highlighting their features and…

Building an LLM-Powered Job Classification Pipeline for Healthcare Recruiting

Upstate NY

See a live demo of an LLM pipeline classifying and structuring thousands of healthcare jobs weekly, using OpenAI…