Quick Start

Hello world

This is a Doc in Progress

Create your first flow: Linkedin Engagement

Datagen is a platform to let you build powerful AI workflow effortlessly. Let’s start with a simple example:

Find people who has engaged with your target Linkedin account’s recent posts

By end of this quickstart, you will have a working API that for any target Linkein URL you will get back a unique set of people who has commented on target’s post along with the number of posts they have commented on. So you can quickly see who has been the high engager. Let’s see how easy can it be.

Create a Flow

Once login, you have the dashboard of all your flows. To create your first flow, click on the + New Flow button. Put in the name of your flow and hit Enter. And for the purpose of this walkthrough, we will name it as

Linkedin Engagement.

Add Steps

Great job for creating your first flow! Now you can start to create your first Step.

Three important designs of Step in Datagen:

1. No Pre-built Nodes : Just describe what you want to do and Datagen will try to create the code for you. No more dauting learning of the platform.
2. Step Chaining : You can chain multiple steps together to create and experiment with a complex flow. No need to provide a mega prompt up front.
3. Code as Guardrail : Instead of asking LLM to calculate 2+2=4 , we ask it to build a function to do sum(a+b). It turns the stochastic LLM output to deterministic workflow. No more AI going rogue, and much cheaper and faster to handle large scale data.

1. Step 1: Get the target Linkedin posts

   Here is an exmple prompt to get the target Linkedin posts:

   given @target_linkedin_url, get me the last 10 posts of the target Linkedin account using /get_linkedin_person_posts and save the result to #posts
   

   1. Copy the prompt above and paste it into the step description
   2. Hit the Submit button to create the step.
   3. Right after you submit the step, you can soon see the input section on the right is populated with the input that you put @ in front. In this example, we have @target_linkedin_url as the input.

   

   4. Now simply put in the target linkedin url and click Build to create the step. For this example, you can try

   https://www.linkedin.com/in/jordancrawford/
   

   (Jordan Crawford is a thought leader in AI and GTM)

   5. After you click Build, you will see the thinking section on the right to show our agent building the step in real time. and once the step is built, you can see the output in the output section on the right. And because you put #posts as the output, you can see the output is saved to the posts variable.

   

   6. Finally, the reason we can access the post data is because we call the tool get_linkedin_person_posts. You can easily select the tool by typing / in front of the tool name.
2. Step 2: Get the commenters of the posts

   Here is an exmple prompt to get the commenters of the posts:

   for each post in @posts, find all the unique commenters 
   using /get_linkedin_person_post_comments and save the result to #commenters
   with their first name, last name, linkedin url and no of post they have commented on. 
   

   1. Copy the prompt above and paste it into the step description, follow by the Submit and Build button.

   2. See how we reference the previous step’s output #posts by putting @posts as the input in step 2. And this is how datagen can chain multiple steps together with consitetent data flow.

   3. Unlike Step 1 with simplet data pull, Step 2 handles

      • loop through the posts
      • find all the unique commenters.
      • Count the number of posts each commenter has commented

      with just one simple prompt. we get the output like below:

   

   4. You can also see the code generated from LLM in the code section on the right.

   

Deploy the Flow

Congrat to build your first flow! Now lets let it run whereever we want!
With single click, Datagen will deploy the flow as an API endpoint. So you can use it anywhere you want.

1. Deploy the Flow

   Click the Deploy button on the top right.

   

   And if success, you should see a pop up like below:

   
2. Get the Deployed Flow

   First hit the little ‘cube’ icon on the left menu bar. Then just click the deployed flow you just created.

   
3. Access deployed API

   You can see the deployed API endpoint by clicking the API tab in deployed flow page.

   In the API section, you can see the input variables target_linkedin_url we set in the flow that can be configured to whichever target linkedin url you want.

   

   To access DATAGEN_API_KEY, simply click the generate a key link and copy the key.

   
4. Use the API

   To test the API, you can open your terminal or any API client you like by sending a POST request to the endpoint with the API key. First, export the API key to an environment variable.

   // An example of using curl to test the API in terminal
   // First Export the API key to an environment variable
   export DATAGEN_API_KEY=<your_api_key>
   

   Then, copied from the curl example section in API, and change the <string> to your target url.

   Example below(for my API endpoint):

   // Copy the curl example and change the `<string>` to your target url. 
   curl -X POST https://leadgen-server.fly.dev/apps/c70b390d-2c66-46a9-a2ec-58d19ec1d786 \
   -H "X-API-Key: $DATAGEN_API_KEY" \
   -H "Content-Type: application/json" \
   -d '{"input_vars":{"target_linkedin_url":"https://www.linkedin.com/in/jordancrawford/"}}'  
   

5. Integrate the API to your preferred platform

   Following the example above, you can see the API is very simple to use. While we are continuing to build the integration with more platforms, here are few common resouces to integrate the API:

   • Google Sheet: Through API connector, you can easily integrate the API to your google sheet.
   • Notion: Using Note API connector to send API data to your Notion.
   • MCP: Using MCP to run your flow through any MCP supported Client, such as Cursor, Claude, and more.
   • Clay: Native API integration with Clay to enrich your table with Datagen data.
   • n8n: n8n is a popular open source automation platform that you can use to integrate the API to your preferred platform.
   • AirOps: Native API integration with AirOps to enrich your data with Datagen data.

Pro Tips:

Datagen leverage the power of LLM to build the step into code on the fly. It empowers great capability especially with complex data transformation without the need to manually write code. That being said, we still have few tips on Step creation to help you get the best out of Datagen:

Tips No. 1: Prompt it like ChatGPT

Just like working with any LLM, the more specific you are, the better the result you will get. For example, instead of saying, ” Give me LinkeIn engagement”, try to define what is the egnagement you want like “Find me the commenter who has commented on target Linkedin’s last 10 posts”.

Tips No. 2: Breakdown the step into smaller steps

When in doubt, break down the step into smaller steps. Instead of requiring a mega prompt up front, you can build and experiment with small steps first. This not only improves the success rate of the step creation, but also allows you to have observability on the intermediate results.

Tips No. 3: Decouple the data fetching and data processing

In general, LLM is pretty good to pull the data but may sometime struggle with complex data processing. By Pulling the data first, you can save both time and money to repull the data again in later steps.

Tips No. 4: Use the right tool

Datagen has a rich set of tools to help you pull the data you need. Use the right tool to get the data you need.

We want to hear from you!

Whether you feel stuck due to our still-in-progress design or want to share how things should be done. We are here to listen! Join our Discord or book a time with us to chat Here!

Explore More

Tools in Datagen

See more details about the available tools in Datagen

Templates

Want more examples? Check out our ready-to-use templates!