Table of Contents
Artificial Intelligence Translates the Language of Animals
Kim Kyung-jin, Attorney at Law
The Story of AI Learning to Listen to Whales, Dolphins, Birds, and Bees
Twelve chapters on how AI listens to dolphins, sperm whales, humpback whales, birds, and bees to find rules in their sounds, and what this technology means for its risks and for animal rights. Written in simple sentences a child can read, with verified sources in every section.
Table of Contents
AI Deciphers Ancient Scripts
Kim Kyung-jin, Attorney at Law
Ancient Records Revived by AI
In twelve chapters, this book explains how AI revives records once unreadable, from burned scrolls and wooden slips buried in mud to broken clay tablets. It covers virtual unrolling at Herculaneum, virtual collation of oracle-bone texts, reading Silla wooden tablets, computational analysis of undeciphered scripts, and multispectral archives, with verified references for each chapter.
Table of Contents
Artificial Intelligence for New Materials Design and Rocket Propulsion Engineering
Kim Kyung-jin, Attorney at Law
AI Potentials, Self-Driving Laboratories, and Physics-Informed Machine Learning (PIML)
Ten chapters on how artificial intelligence is changing new materials and rocket propulsion: atomic simulation, generative models, self-driving labs, high-temperature alloys, metal 3D printing, combustion, cooling design, and engine diagnosis and control. Written without equations, with verified sources in every chapter.
AI Library
The Age of Autonomous Scientific Discovery
Kim Kyung-jin, Attorney at Law
AI Scientists and Self-Driving Labs
This book follows how AI scientists and self-driving labs are changing the way science generates and verifies claims. It covers literature-based discovery, natural-language protocols translated into robot commands, multi-agent research systems, closed-loop laboratories, materials search, the verification gap, chains of evidence, research harnesses, journal ethics, and legal responsibility.
AI Library
A New Era of Life Sciences Opened by Artificial Intelligence
Structural Proteomics, Genomic Foundation Models, Autonomous Laboratories, and Global Governance
Kim Kyung-jin, Attorney at Law
This book is a research volume compiled with artificial intelligence. A human selected the materials and structured the work, while AI models drafted the sentences and cross-checked the facts.
AI Library
The Double Structure of Digital Sovereignty
Europe’s Departure from Palantir and the Chains of American Big Tech
Kim Kyung-jin, Attorney at Law
This is a record of 2026, when European intelligence agencies and defense ministries began removing analytics tools from America’s Palantir. It covers the replacement decisions made by France’s General Directorate for Internal Security (DGSI), Germany’s Federal Office for the Protection of the Constitution (BfV), and the Netherlands Ministry of Defense; the incident in which US export controls severed an ally’s ac…
New English Edition
Artificial Intelligence in Horticulture
Kim Kyung-jin, Attorney at Law
Across five chapters and ten sections, this book examines computer vision for crop diagnosis, harvesting robots and autonomous field systems, smart greenhouses and digital twins, precision irrigation and supply-chain quality control, high-throughput phenotyping, and predictive breeding.
New English Edition
Artificial Intelligence in Food Crop Agriculture
Kim Kyung-jin, Attorney at Law
Across six chapters and eighteen sections, the book examines digital agricultural infrastructure, remote sensing, crop diagnosis, yield forecasting, precision irrigation, genomics, molecular breeding, agricultural robotics, climate-smart agriculture, and global food security.
New English Edition
The Future of Forestry and Agroforestry
Kim Kyung-jin, Attorney at Law
Driven by Artificial Intelligence and Digital Innovation
Across five chapters and fifteen sections, the book follows satellites, drones, LiDAR, digital twins, forest-specific language models, wildfire and pest forecasting, forestry robotics, agroforestry, timber traceability, and forest carbon markets.
New English Edition
Smart Livestock Farming: AI Enters the Barn
Kim Kyung-jin, Attorney at Law
Sensors listen, cameras watch, and artificial intelligence helps farmers decide.
Across five chapters and fifteen sections, the book follows precision livestock farming from animal health and reproduction to robotic milking, virtual fencing, digital twins, methane reduction, welfare, and data ownership.
Table of Contents
Han Dong-hoon, Busan Buk-gu Gap: A Record of the 100 Days Before and After the Election (Mar. 26-Jul. 3, 2026)
Kim Kyung-jin
Table of Contents and 13 sections
From March 26 to July 3, 2026, this record follows the spring after expulsion, the Busan Buk-gu Gap by-election, victory as an independent, and the first bill submitted in the National Assembly.
Table of Contents
Artificial Intelligence and Medicine
Kim Kyung-jin, Attorney at Law
AI in clinical care, hospitals, education, and research
AI in medical imaging, risk prediction, treatment planning, hospital operations, education, and research, with patient safety, privacy, and accountability.
[AI Library] Chapter 17: Job Data Web Scraping Workflow
Mastering Claude Code
Chapter 17: Job Data Web Scraping Workflow
Kim Kyung-jin
Mastering Claude Code
Job Design Combining Firecrawl MCP and Planning Mode
The left side of the screen displays remote work job listings related to social media, packed tightly together. 622 listings. That comes to 21 pages. If you tried to copy this information by hand one by one, you would be working all day and still not finish. This is where the agent workflow shines.
This chapter follows the process of combining a tool called Firecrawl with Claude Code's planning mode to automatically gather and organize large-scale job data. Through hands-on practice, you will come to understand the practical power of MCP connection servers.
What Is Firecrawl
Firecrawl is a tool that converts websites into LLM-friendly data. Beyond simple web scraping, it extracts website text in markdown format, captures screenshots, maps site structure, and performs data crawling.
If you put a URL into the McDonald's website and request web scraping, it brings back all the text from that page in markdown form. But this is only a tiny fraction of what Firecrawl can do. Various capabilities like scraping, searching, extracting, and crawling are integrated into a single tool.
The Principle of Connecting Tools with MCP
A crucial concept appears here: MCP, or Model Context Protocol. To understand this concept, think of Gmail. Within Gmail exist many individual actions like sending email, drafting messages, importing email, and more.
MCP wraps all these actions into a single server, allowing the agent to judge on its own which tool to use, when to use it, and what parameters to fill in.
To explain it as a cake-making analogy: instead of going to the egg shop when you need eggs, the flour shop when you need flour, and the candy shop when you need frosting, you simply grant access to the supermarket MCP. "Get whatever ingredients you need from the supermarket yourself." That is the essence of MCP.
[Figure 17-1] Conceptual diagram of MCP connection server: architecture for accessing multiple tools through a single connection]
Installing Firecrawl MCP Connection Server
Firecrawl's official documentation provides instructions for installing the MCP connection server. It includes how to run it from Claude Code, so you can copy the command and pass it to Claude Code.
During installation, you will see a prompt asking you to enter an API key. An API key is a kind of password. Entering it directly into Claude Code's conversation history is not advisable for security. Instead, use a .env file to store it.
Claude Code creates a placeholder for the API key in the .env file. You then paste the API key issued from your Firecrawl dashboard directly into that spot and save it. Firecrawl offers 500 credits for free signup, which is plenty for practice.
One thing to watch out for: when Claude Code runs Bash commands, the API key can be exposed in the conversation history. If it is a free key, this is not a major problem, but for sensitive keys, it is safer to get only the installation method from Claude Code and run the actual command directly in a separate terminal.
Designing Tasks in Planning Mode
With installation complete, we move into building the actual workflow. After clearing the conversation history (/clear), switch to planning mode. The reason for starting in planning mode is clear. The agent thinks more deeply, surveys the entire project folder, and raises questions you had not considered.
Copy the URL of the job listings page and paste it into Claude Code, explaining your requirements in natural language.
Claude Code receives this message and checks whether it can use the Firecrawl MCP connection server. Then concrete questions come back. Should it scrape individual job detail pages, where should the Excel file be saved, are there filtering conditions, and so on.
When you answer these questions, Claude Code presents a comprehensive execution plan. It will create a tool called scrape_daily_remote, build a workflow called scrape_job_listings, and perform the actual web scraping.
[Figure 17-2] Example of a web scraping plan presented by Claude Code in planning mode]
Organizing 622 Job Listings into Excel
When you approve the plan and select auto accept mode, Claude Code generates a to-do list and executes it one item at a time.
Checking Execution Results
When the job finishes, the results appear. 209 job listings were collected. Various metrics and location information are organized together. Looking at the file explorer on the left, you can see that the scrape_daily_remote_jobs tool was created in the tools folder and the scrape_job_listings workflow was created in the workflows folder.
Let me explain why this structure matters. The next time you request the same type of web scraping, Claude Code reuses the tools and workflows it has already built. If there were mistakes, it automatically fixes and updates them. This is the self-improvement loop of the WAT framework.
Excel File Structure
Let us open the Excel file saved in the temp folder. Claude Code even added filtering functionality on its own. The column structure is as follows.
[Figure 17-3] Actual screenshot of the generated Excel file]
209 job listings are organized cleanly. Work that would have taken half a day by hand was completed by Claude Code in minutes.
The Importance of Context Management
Let me share a practical tip. As you work, you will see a message at the bottom of the screen that says "45% of context remains." There is a phenomenon called context rot. As conversation history grows longer, AI model performance declines. Before context usage exceeds 60%, it is good to use the compact function to compress the conversation.
Doing this allows you to maintain important information while cleaning up unnecessary records.
Filtering and Regional Classification: Case of Extracting 372 European Sales Positions
This time we raise the difficulty a notch. This is a task to extract only those that meet specific conditions from 214,000 total job listings on a page with no search filter.
Communicating Requirements
This time we intentionally use Bypass Permissions mode. We skip planning mode and throw relatively vague instructions at the agent to observe what results.
Claude Code recognizes the web scraping tool it previously created. After deciding to itself "I can use the scraping tool I made earlier," it formulates a plan. It will scrape sales job listings, filter by Europe, and export to Excel.
Collision with Reality and Adaptation
But an unexpected situation unfolds. There are only 52 European sales positions. The total is 409 sales jobs, but limiting to Europe falls far short of the 500-position target.
This is where the agent's judgment capabilities become apparent. Claude Code automatically recognizes problems and presents options to the user.
If you answer "Please add US sales positions too," the agent continues the task. In this process, temporary files are generated. Intermediate data files like all_sales_jobs and sales_jobs_raw, along with three Python filtering scripts, are created in the temp folder. This is when the separate temp folder structure set up at the project start proves its value.
Final Results
Ultimately, 372 sales position job listings are collected. The Excel file includes a Region column added to the existing fields. Filtering by this column allows immediate classification of job listings from the US, Europe, and worldwide. Selecting Europe alone leaves approximately 49 listings.
[Figure 17-4] Sales position Excel file with regional filters applied]
Handling Unexpected Requests: Generating a US Dentist Prospect List
The true test of an agent workflow is an unexpected request. Until now, the work has been relatively standardized, pulling job listings from a specific URL. This time, a completely different type of request is made.
A Different Type of Request
This request contains no URL. There are no hints about where to get the data. The agent must find the data source on its own.
The Agent's Problem-Solving Process
Claude Code first reviews existing tools and workflows. It then uses Firecrawl to search for a "dentist directory." While it discovers the ADA (American Dental Association) site, that site uses JavaScript for dynamic rendering, so static web scraping does not work.
It does not give up. It explores other options and determines that Yellow Pages is effective. Upon discovering that New York City alone has information on 3,000 dentists, it begins building a new web scraping tool.
However, the first attempt extracts only two dentists. The parsing pattern has a problem. The agent immediately detects this error and corrects the regular expression (Regex) pattern. It updates the tool file itself to prevent the same issue from recurring.
[Figure 17-5] The agent detects a web scraping error and fixes the tool]
What Self-Healing Actually Looks Like
This process shows what self-healing actually looks like. In traditional automation, when an error occurred, a developer had to read logs, fix code, and revalidate. In an agent workflow, the agent performs all these steps itself. It finds errors, analyzes causes, fixes code, and revalidates.
Final Deliverable
Running with the corrected tool yields 120 unique dentist leads across four major cities. The Excel file is structured as follows.
A new workflow (scrape_dentist_leads) and a new tool (scrape_dentist_leads) are automatically created. The next time a similar request comes in, this workflow and tool will be reused.
Multi-Agent Strategy
Here, I introduce an advanced strategy. Claude Code allows opening multiple agents simultaneously. You can have five agents each attempt a different approach, then keep only the workflow that produces the best results and delete the rest. This approach allows parallel validation of which data source is most reliable and which web scraping strategy is most effective.
What this exercise confirms is clear. If you know what form your desired output should take, you can delegate the technology stack and specific implementation details to the agent. Whether it is 622 job listings, 372 sales positions, or 120 dentist leads, tangible results emerge simply by stating your goal in natural language. But what pitfalls do people commonly fall into when using this powerful tool? Let's explore that next.
AI Expert and Attorney Kim Kyung-jin
Specialist in AI legal policy, former member of parliament, author of numerous works
If this book has stayed with you even briefly, please support the next story so it can reach the world.
(Voluntary donation account: Nonghyup 302-1096-0948-81, Account holder: Kim Kyung-jin)
Kim Kyung-jin
Attorney · Former Member of the National Assembly · AI Policy Researcher
© 2026 Kim Kyung-jin. All rights reserved.