AI-Powered Hacker News Job Scraper & Structurer
Integrates with:
Overview
Unlock Automated Job Lead Generation with this AI Agent
Tired of manually sifting through Hacker News 'Who is hiring?' threads? This n8n AI Agent takes over, automatically finding relevant posts, fetching individual job comments, and then using the power of OpenAI (GPT-4o-mini) to perform its 'intelligent data extraction' and 'data structuring' abilities. It intelligently pulls out key details like company name, job title, location, salary, and more, transforming unstructured text into a clean, actionable dataset in Airtable. Stop wasting time on manual data entry and focus on leveraging the insights.
Key Features & Benefits
- Automated HN Scraping: Regularly fetches the latest 'Who is hiring?' posts and their job comments via Algolia and Hacker News APIs.
- AI-Powered Data Extraction: Leverages OpenAI's GPT-4o-mini to accurately parse and extract structured information like company name, job title, location, employment type, salary, and application URLs from raw job postings.
- Structured Data Output: Organizes extracted data into a predefined schema, ready for direct import into Airtable or other systems.
- Time-Saving Automation: Frees you from hours of manual searching and data entry, allowing you to focus on using the data.
- Customizable & Extensible: Easily modify the OpenAI prompt, data schema, or integrate with other tools in your n8n instance.
- Lead Generation & Market Insights: Perfect for recruiters, job seekers, sales teams tracking company growth, or market researchers.
Use Cases
- Automated sourcing of tech job leads for recruitment agencies.
- Identifying fast-growing companies hiring specific roles for B2B SaaS sales prospecting.
- Solopreneurs and founders monitoring hiring trends and competitor activity.
- Market researchers collecting and analyzing job market data from a key tech community.
- Individuals looking for new job opportunities by aggregating relevant HN listings.
Prerequisites
- An n8n instance (Cloud or self-hosted).
- OpenAI API Key with access to a model like gpt-4o-mini.
- Airtable account and Personal Access Token.
- Algolia Search API Key (for the 'Search for Who is hiring posts' node credential).
Setup Instructions
- Download the n8n workflow JSON file.
- Import the workflow into your n8n instance.
- Configure the 'Search for Who is hiring posts' (HTTP Request) node: This node uses Algolia to find 'Who is hiring?' threads. Ensure the n8n credential named 'Algolia Auth' (Header Auth type) is configured with your Algolia Search API Key (Header name:
x-algolia-api-key). The Application ID is usually public and pre-filled in the node's query parameters. - Configure the 'OpenAI Chat Model' node: Select your OpenAI credential (containing your API Key) and choose a model (e.g., gpt-4o-mini is pre-selected).
- Customize the JSON schema in the 'Structured Output Parser' node if you need to extract different data fields or modify the existing ones.
- Configure the 'Write results to airtable' node: Select your Airtable credential (Personal Access Token). Then, specify your Airtable Base ID and Table Name. You can use the Airtable base template provided in the workflow's sticky note (https://airtable.com/appM2JWvA5AstsGdn/shrAuo78cJt5C2laR) or map to your existing base structure.
- (Optional) The 'Limit for testing (optional)' node is set to 5 items. Adjust or remove this node for full runs.
- Activate the workflow. It is triggered manually by default; consider changing to a cron schedule (e.g., monthly) for ongoing automation.
Want your own unique AI agent?
Talk to us - we know how to build custom AI agents for your specific needs.
Schedule a Consultation