How to Build a Social Media Scraper AI Agent with n8n! (Step by Step)
Introduction
Imagine being able to track every post your competitors make
across LinkedIn, Instagram, YouTube, and even X in seconds. That's exactly what
the Social Media Scraper AI engine does. I place the brand social link and
within moments, it collects all their latest posts along with captions, [music]
posting dates, transcripts, and engagement date automatically saved in your
Google sheet. This is perfect for creators, marketers, and founders [music] who
want to study what's performing well without
wasting hours scrolling through feeds. And the best part,
you can build the same automation yourself using N8 without a single line of
code. So, in this video, I'll show you how to build and run a social media
scraper AI agent using NAN. And I'm Brian from website learners and let's get
started.
Step 1: Get the Workflow
To create a social media scraper AI agent, we are going to
do four steps. The first step is to get the workflow. To get the workflow, just
click the link in the description. Click download and the social media scraper
AI agent
workflow will be downloaded. In order to download this
workflow, you need to have the en hosting on Webspace Kit. Since I've already
logged in, it'll download directly. You can watch this video to [clears throat]
learn how to set up the nan hosting on Webspace Kit. All right. Once you got
the hosting and the workflow, go to your nan account. Then click here, select
import from file and upload the workflow file you just downloaded. As you can
see here, we've got our workflow. All right, we have
successfully imported our workflow. Now we need to set up
the nodes and connect the required API services to our workflow.
Step 2: Connect Apify to N8N
Getting the Apify API Key
Now let's go to the second step which is to connect ampify
to nin. Appify is a web scraping tool and here we are using it to pull data
from different social media platforms. to connect ampify. Let's open this node.
Scroll down and this is where we need to add the appy API key. To get the API
key, click the link in the description and it'll take you to this page. Let's
click get
started and sign in with your Google account. Now, click
here and go to billing here. Click upgrade and choose the plan that best suits
your needs. I'm going to select the starter plan. Fill in these details. Click
save and continue. Select a payment method. Enter your payment details and
click save and continue. All right. Now, click here. Search for API. Click go
to API tokens. And here you can see the API key we need. So let's copy it. Go
back to NAN and paste it here.
Applying API Key to All Nodes
Now go back and in the same way paste the API key in
these nodes too. So let's open them one by one and paste it
in this field. Perfect. We have successfully connected appi to NA10.
Step 3: Connect Google Sheets to N8N
Setting Up Google Sheets Credentials
Now let's move on to the third step which is to connect
Google sheets to n10. Google sheets is where all our scrape data will be stored
so that we can access them easily. So to connect Google sheets to n open this
node, click here and select create new credential. Here we need to enter Google
credentials. And if you don't know how to generate these credentials, watch
this video where I'll show exactly how to set them up for
Google Sheets, Google Drive, Google Docs, and other Google apps.
Preparing the Google Sheet Template
Okay, once you've added the credentials, click here and sign
in with your Google account and click continue. Next, click the Sheets link in
the description and it'll open this template. Click on file. Click on make a
copy and click on the make a copy button again. Now let's go back to N8 and
select the Google sheet you just created. Now click on the sheet drop
down and select LinkedIn scrape. All right. Now Google sheet
is connected to any but since we are scraping data from multiple social media
platforms each platform has its own Google sheet node.
Assigning Sheets for Each Platform
This means we need to select the correct sheet for each one
of them to make sure the scrape data is added to the right place. So let's
start with this node. Just open it and select the YouTube scrape sheet.
Next, open these nodes and select the Instagram real scrape
sheet. And finally, open these nodes and select the X scrape node. Once that is
done, the Google sheet will be successfully connected to NA10.
Step 4: Connect OpenAI to N8N
Adding OpenAI API Key
Let's go to the fourth step which is to connect OpenAI to
NAN. Open AI is used to generate the transcripts of Instagram res and YouTube
videos. These transcripts are then automatically organized in the Google sheet.
So to connect OpenAI to NA, open this node, click here and select create new
credentials.
Now, this is where we need to enter the API key. To get the
API key, just click the link in the description.
Now, go to API keys. Click create new secret key here. Give
a name for your key and click create secret key. As you can see, we now have
our API key. But keep in mind, you'll only be able to get an API key from
OpenAI if you've added a payment method first.
Connecting Additional OpenAI Nodes
Now, let's copy this key. Paste it in NAN. Since we have two
more OpenAI nodes to configure them, just open them one by one and they'll be
connected to NA10. Great, we have successfully connected open air to n. Just
click on save to store your workflow.
Running the Workflow
Scraping LinkedIn
So now we have connected all the required API services with
the nent workflow and it's ready to run. Just click on execute workflow. Now
paste the social media link of any competitor. Let's start with the LinkedIn
link. I'll copy this brand's profile.
come back and paste it here. Once you send it, the workflow
starts running. When it's done, open your Google sheet. And here it is. The AI
agent has scraped all the recent LinkedIn posts along with the posting dates,
captions, and even the follow count.
Scraping Instagram
Next, let's try with Instagram. Just copy the Instagram
account link, send it here, and once it is processed, go to your Google sheet
and switch to the IG real scrape tab. At the bottom, you'll see the recent
reals with posting dates, captions, transcripts, likes, and hashtags.
Scraping X
Let's test it with the Xlink. Just copy the X account link,
send it here.
Go to your Google sheet and switch to the Xcrape tab at the
bottom. And here's all the scrape data neatly organized.
Scraping YouTube
Finally, let's test YouTube. Copy the channel link, send it
here, and once it is done, go to your Google sheet and switch to your YouTube
tab.
and you'll see every recent video along with a title, date,
likes, view, and a subscriber count along with a transcript and caption. Pretty
cool, right?
Conclusion
Now, you can use all this data to research competitors, find
content ideas, and plan your next campaign smarter. So, that's it, guys. This
is how you can build and run your social media scraper
automation using NA10. Now, you can start building this
workflow for yourself. The first step is launching NA10 on Webspace Kit. And to
do that, just click here and it'll take you to the same page that we saw in the
beginning. So go ahead and start creating workflows.
Also, you can get our latest videos directly on WhatsApp.
You can just scan the QR or tap the link and hit follow. And that's it. Thanks
for watching. I'll see you guys in the next one. Until then, take care.
Bye-bye.
Post a Comment