AI Agent Toolkit: Web Scraping & API Interaction Examples

Build Powerful AI Agents with Custom Tools

This n8n workflow provides a practical toolkit with two distinct AI Agent examples, demonstrating how to empower your automations with intelligent web scraping and dynamic API interaction capabilities. The first agent example uses Firecrawl to extract content from webpages based on natural language instructions. The second example interacts with the Bored API to suggest activities, showcasing how AI can select and call external services with appropriate parameters. Both leverage OpenAI for reasoning and the n8n toolHttpRequest node for simplified API integration, offering a blueprint for building sophisticated AI-driven automations.

Key Features & Benefits

• Dual Agent Examples: Provides clear, working examples of an AI web scraping agent and an AI API interaction agent.
• Intelligent Web Scraping Ability: Learn to build agents that fetch and parse main content from webpages using services like Firecrawl, guided by LLM instructions.
• Dynamic API Tool Use Ability: Demonstrates how AI agents can select and call external APIs (e.g., Bored API) with parameters derived from user requests.
• OpenAI-Powered Reasoning: Utilizes OpenAI's language models for understanding queries, selecting appropriate tools, and formulating API calls.
• Simplified Tool Integration: Highlights the toolHttpRequest Langchain node, drastically reducing workflow complexity for creating API-enabled AI agents.
• Foundation for Custom Agents: Use these examples as a blueprint to build your own AI agents tailored to specific data gathering and task automation needs.

• An n8n instance (v1.47.0 or newer, Cloud or self-hosted, for toolHttpRequest node compatibility).
• OpenAI API Key with access to a suitable model (e.g., gpt-3.5-turbo or gpt-4).
• Firecrawl API Key (for the web scraping agent example).
• (Optional) Credentials for other APIs you wish to integrate with your custom agents.

1. Download the n8n workflow JSON file.
2. Import the workflow into your n8n instance.
3. Configure the 'OpenAI Chat Model' and 'OpenAI Chat Model1' nodes: Select your OpenAI credential or enter your API Key directly in the node parameters.
4. For the Web Scraping Agent (using Firecrawl): a. In your n8n instance, go to 'Credentials'. b. Create a new credential or locate the one named 'Firecrawl API' (as referenced in the 'Webscraper Tool' node). c. Set the credential type to 'Header Auth'. d. Add one header: Name = Authorization, Value = Bearer YOUR_FIRECRAWL_API_KEY. Save the credential. e. Ensure this credential is selected in the 'Webscraper Tool' node.
5. For the Activity Suggestion Agent: The 'Activity Tool' node uses the public Bored API and requires no specific credentials by default. You can adapt this tool to call other APIs by changing its URL and parameters.
6. Review the 'Set ChatInput' and 'Set ChatInput1' nodes to see example prompts. Modify these to test different scenarios with each agent.
7. (Optional) Customize the toolDescription in the 'Webscraper Tool' and 'Activity Tool' nodes to refine how the AI understands their purpose and parameters.
8. (Optional) Add more toolHttpRequest nodes connected to the 'AI Agent' or 'AI Agent1' nodes to expand their capabilities with other APIs.
9. Activate the workflow. Trigger it using the 'When clicking ‘Test workflow’' node to see the agents in action.