Get Google Trends data with Raspberry PI and Python

Last Updated on 2nd September 2023 by peppe8o

In this tutorial I’m going to show you how to get Google Trends data with a cheap Raspberry PI and Pytrends library.

For bloggers and website owners, organic trends are all. Search results show people interests and drive content production. Being first to write about a topic can be a great SEO advantage. Getting Google Trends data with Raspberry PI and Pytrends can assure you this advantage.

Pytrends is a Python library able to interface Google Trends API (Application Programming Interface) and gettind Google search trends data. It is important to know that it isn’t a Google official API, so it is based on project maintainers. This also means that if Google changes its API backend it may take time to re-align new big-G API structure,

Pytrends allows simple interface for automating reports download which can be managed with common python tools. Crawled data can be scheduled via Cron and then sent via email with common line-command clients like Mutt.

Pytrends works with both Python 2.7+ and Python 3.3+ and requires Requests, lxml, Pandas.

In this guide I’ll use a Raspberry PI Zero W, but this will work with all other Raspberry PI models with internet connectivity.

What We Need

As usual, I suggest adding from now to your favourite e-commerce shopping cart all the needed hardware, so that at the end you will be able to evaluate overall costs and decide if to continue with the project or remove them from the shopping cart. So, hardware will be only:

• Raspberry PI Zero W (including proper power supply or using a smartphone micro usb charger with at least 3A) or newer Raspberry PI Board
• high speed micro SD card (at least 16 GB, at least class 10)

Check hardware prices with following links:

Step-by-Step Procedure

Prepare Operating System

To start preparing your OS, install Raspberry PI OS Lite (for a fast, headless OS). You can also use Raspberry PI OS Desktop (working from its internal terminal).

Make your OS up to date. From terminal:

sudo apt update -y && sudo apt upgrade -y

We have now to install required packages. We need pip to install pytrends. But pip includes a NumPy version not working with Raspberry PI Python version. The trick is uninstalling pip NumPy and installing it again with apt. From terminal:

sudo apt install python3-pip
pip3 install pytrends
pip3 uninstall numpy
sudo apt install python3-numpy

If you want to send crawled data by email, you can also find usefull my install Mutt article.

Use Pytrends in a Python Terminal Session

Before digging in a script which automated data collection and email delivery, let’s deep what Pytrends can do. We’ll use pytrends in a Python session, showing queries and results. I’ll look for a single keyword (Ferrari), but you can also use multiple keywords.

Before all, from terminal start a Python (3) session by typing:

python3

Your terminal will enter an interactive Python session:

pi@raspberrypi:~ $ python3
Python 3.7.3 (default, Jul 25 2020, 13:03:44)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

Import pytrend library:

from pytrends.request import TrendReq

and open a pytrend session:

pytrend = TrendReq()

Download your keyword related data with following command:

pytrend.build_payload(kw_list=['Ferrari'])

you can include multiple keywords using “kw_list=[‘Ferrari’, ‘Formula 1’, ‘Valentino Rossi’, ‘MotoGP’]”.

At this point, data can be printed in your terminal console whit a few of commands depending on what you are looking for. Following will show commands runned on terminal and relative results. Commands will be marked with a green char only to distinguish it from results.

Please note that some commands are keyword-related, while other ones are absolute. Data must be read according to Google Trends interpretation.

>>> print(pytrend.interest_over_time())
Ferrari isPartial
date
2015-12-13 50 False
2015-12-20 57 False
2015-12-27 52 False
2016-01-03 52 False
2016-01-10 50 False
… … …
2020-11-01 40 False
2020-11-08 43 False
2020-11-15 43 False
2020-11-22 44 False
2020-11-29 42 True
[260 rows x 2 columns]

>>> print(pytrend.interest_by_region())
Ferrari
geoName
Afghanistan 0
Albania 0
Algeria 7
American Samoa 0
Andorra 0
… …
Western Sahara 0
Yemen 0
Zambia 0
Zimbabwe 0
Åland Islands 0
[250 rows x 1 columns]

>>> print(pytrend.related_queries())
{'Ferrari': {'top': query value
0 ford ferrari 100
1 lamborghini 66
2 458 ferrari 60
3 ferrari car 60
4 ferrari f1 57
5 f1 56
6 ferrari vs ford 53
7 ferrari price 50
8 488 ferrari 41
9 la ferrari 40
10 ferrari enzo 39
11 ferrari la ferrari 37
12 ferrari for sale 33
13 porsche 30
14 ferrari california 27
15 ferrari cars 25
16 ferrari world 24
17 ferrari spider 24
18 f40 ferrari 22
19 ford v ferrari 22
20 ferrari 2019 21
21 mercedes 21
22 ferrari italia 21
23 ferrari 2017 20
24 ferrari f12 20, 'rising': query value
0 ferrari 812 superfast 49000
1 ferrari 812 45050
2 ford vs ferrari movie 32700
3 ferrari 2019 29400
4 ford v ferrari 25000
5 ferrari gtc4lusso 24300
6 ferrari sf90 24250
7 ken miles 23450
8 ferrari f8 tributo 21450
9 ferrari 488 pista 21250
10 ferrari f1 2019 18000
11 aaja na ferrari mein 16750
12 portofino ferrari 14600
13 ford vs ferrari cast 14050
14 ferrari 2018 13950
15 ferrari vs ford 13400
16 ferrari f1 2018 12700
17 ferrari f1 2020 11550
18 ferrari monza sp2 11050
19 ferrari gtc4 lusso 9900
20 ferrari sf90 stradale 9750
21 knives out 8350
22 ford против ferrari 7850
23 ferrari 2020 7350
24 ford vs ferrari netflix 6350}}

>>> print(pytrend.trending_searches())
0
0 Gunner olszewski
1 Seahawks
2 Logan Paul
3 Rudy Giuliani
4 Chicago Bears
5 Dolly Parton
6 Alabama football
7 Drew Lock
8 Browns vs Titans
9 Kelly Loeffler
10 Pumas vs Cruz Azul
11 Jets
12 Attack on Titan Season 4
13 Texans
14 Tottenham vs Arsenal
15 Drew Brees
16 Shameless
17 Liverpool vs Wolves
18 Washington Football Team
19 Pearl Harbor Day

>>> print(pytrend.today_searches())
0 Gunner olszewski
Name: query, dtype: object

>>> print(pytrend.top_charts(2018, hl='en-US', tz=300, geo='GLOBAL'))
title exploreQuery
0 World Cup
1 Avicii
2 Mac Miller
3 Stan Lee
4 Black Panther
5 Meghan Markle
6 Anthony Bourdain
7 XXXTentacion
8 Stephen Hawking
9 Kate Spade

>>> print(pytrend.suggestions(keyword='Ferrari'))
[{'mid': '/m/02_kt', 'title': 'Ferrari S.p.A.', 'type': 'Car manufacturer'}, {'mid': '/m/0179v6', 'title': 'Scuderia Ferrari', 'type': 'Professional sports team'}, {'mid': '/m/0h5ww88', 'title': 'Ferrari', 'type': 'Automobile make'}, {'mid': '/m/08jxr8', 'title': 'Franck Ribéry', 'type': 'French footballer'}, {'mid': '/g/11f6y3pfry', 'title': 'Ford v Ferrari', 'type': '2019 film'}]

Create your Pytrends Script for Automated Data Delivery

If you want to automate data collection and sending via email, you will need to make it all configured in a script. Download pytrends example script from my download area:

wget https://peppe8o.com/download/python/pytrends_example.py

or copy it in a new file from following code:

# Import required libraries
import os
from pytrends.request import TrendReq

# Open file collecting email body data
# print outputs will be redirected to this file
f = open("tmpTrendsData.txt", "w")

# Open a pytrend session (only need to run this once)
pytrend = TrendReq()

# Create payload and capture API tokens. Only needed for interest_over_time(), interest_by_region() & related_queries()
# you can include multiple keywords using kw_list=['Ferrari', 'Formula 1', 'Valentino Rossi', 'MotoGP']
pytrend.build_payload(kw_list=['Ferrari', 'Formula 1', 'Valentino Rossi', 'MotoGP'])

# Interest Over Time
#print(pytrend.interest_over_time(), file=f)

# Interest by Region
#print(pytrend.interest_by_region(), file=f)

# Related Queries, returns a dictionary of dataframes
print(pytrend.related_queries(), file=f)

# Get Google Hot Trends data
#print(pytrend.trending_searches(), file=f)

# Get Google Hot Trends data
#print(pytrend.today_searches(), file=f)

# Get Google Top Charts
#print(pytrend.top_charts(2018, hl='en-US', tz=300, geo='GLOBAL')

# Get Google Keyword Suggestions
#print(pytrend.suggestions(keyword='Ferrari'), file=f)

# Send collected data via email and remove temp file
f.close()
os.system('mutt -s "Today Google Trends data" recipient@example.com < tmpTrendsData.txt')
os.remove("tmpTrendsData.txt")

Note that some queries are commented, so that you can choose which query is interesting for you and sending only those results.

This file differs from previous commands adding the ones to manage date export to a file and sending it as body in email. Prelase find and change in final rows the “recipient@example.com” address with your email address. Remember that we aready setup (and tested) Mutt (please refer to my mutt installation guide, if not already done).

Following line adds os library to our script. This permits us running linux shell command from python:

import os

To make easy composing emnail body from data gatered from pytrend, we create a temporary file on which we’re going to save query output:

f = open("tmpTrendsData.txt", "w")

Please note that all print comands are followed by “, file=f)”: in Python 3 this redirects print output to a file instead of default STDOUT.

When data collecting is finished, file is closed:

f.close()

We now use os library to run shell command with mutt to send email, including file content as body:

os.system('mutt -s "Today Google Trends data" recipient@example.com < tmpTrendsData.txt')

Finally, we remove temporary file as it is sent via email:

os.remove("tmpTrendsData.txt")

Scheduling Script Execution with Cron

Cron is the default tool in Raspberry PI (but, in general, for Unix systems) for configuring scheduled tasks. It is used to schedule commands or scripts to run periodically and at fixed intervals.

You can check running tasks with the command:

crontab -l

In a fresh Raspberry PI os installation crontab is empty. Before adding a new task, we need to understand the 6 components for a cron entry: minute, hour, day of month, month of year, day of week, and the command to be executed. All these components are divided by a simple space character.

# * * * * *  command_to_execute
# ┬ ┬ ┬ ┬ ┬
# │ │ │ │ │
# │ │ │ │ │
# │ │ │ │ └───── day of week (0 - 7) (0 to 6 are Sunday to Saturday, or use names; 7 is Sunday, the same as 0)
# │ │ │ └────────── month (1 - 12)
# │ │ └─────────────── day of month (1 - 31)
# │ └──────────────────── hour (0 - 23)
# └───────────────────────── min (0 - 59)

“*” char is the wildcard.

Step jobs will be identified with “*/” before step amount. For example, a job running every 5 minutes will have following format:

*/5 * * * * command_to_execute

Back to our script, supposing that our script is saved on pi user home, scheduling it once a day, every day at 8:00 am requires following line:

0 8 * * * python3 /home/pi/pytrends_example.py

Add this line to your cron with following command:

crontab -e

Select, if asked, your favourite cron editor (nano in my case) and append your schedulation line to opened file. Save, exit and wait for cron execution to run.

Enjoy!