Python Code for Pulling API Data Easily

Application Programming Interfaces (APIs ) are extremely useful for grabbing data. They let you fetch data from websites, services, and apps programmatically, which means you don’t need to manually copy information and paste it into a spreadsheet or TXT file. 

Python makes this magic feel downright effortless. With its clean syntax and powerhouse libraries (requests, pandas, and json), you can pull live data for dashboards, trend analysis, or automated reports in a handful of lines. Let the language do the heavy lifting while you focus on insights, not buttons and download links.  

In this guide, we’ll show you how to call an API, wrangle the response, and handle common hiccups along the way. We have some examples you can practice with and use concepts from in your own projects. Once you harness these techniques, report-building and dashboard refreshes that once took hours will drop to minutes. 

Getting Started: Python API Request Example

Before you can start coding, you’ll need to set up your environment. We’ll be using Python's requests library, which will communicate with most APIs (if not all).

Installing Python and the Requests Library

If you don't have Python installed yet, download it from python.org. Once you’ve installed Python, you’ll need to add the requests library. Open your command prompt or terminal and type:

pip install requests

Once that finishes, you're ready to make API calls. It's a very easy install and doesn’t need any other configuration steps.

Understanding API Endpoints and Responses

APIs are basically URLs that return data instead of web pages. These addresses (endpoints) accept requests and send back information in structured formats that apps can understand easily.

Most modern APIs use REST architecture and return data in JSON format. JSON is like a series of nested lists and dictionaries that Python handles very well, which is why Python is an excellent choice for this kind of work.

Here's what a basic endpoint might look like:

• https://api.example.com/v1/users: Gets a list of users

• https://api.example.com/v1/users/123: Gets information about user 123

Each API has its own endpoints and rules for using them. If you are trying to make sense of a new API, try checking out the API documentation to find what endpoints are available, and what data they return. 

When there is no documentation, you’ll have to rely on a bit of trial and error to figure out which endpoints you are after. If they are part of a website, you can open your developer tools in your web browser and select your network tab. 

Refreshing the page with your developer tools open will show you the API endpoints the website is using to pull data, which is normally a good starting point for your exploration.

Python Code for Pulling API Data

Below is some actual code for fetching API data. It's quite basic, but it is very easy to understand and use.

Making a Simple API Call with requests.get()

Here's a basic example that pulls data from a public API:

import requests
# Make a GET request to an API endpoint
response = requests.get('https://jsonplaceholder.typicode.com/posts/1')
# Check if the request was successful
if response.status_code == 200:
    print('Request successful!')
    print(response.text)
else:
    print(f'Request failed with status code: {response.status_code}')

This code sends a GET request to fetch a single post from a test API. The response variable contains everything the API returned.

Handling JSON Responses in Python

As we mentioned earlier, most APIs return data in JSON format. Python and JSON are easy to get working together:

import requests
response = requests.get('https://jsonplaceholder.typicode.com/posts/1')
if response.status_code == 200:
    # Convert JSON response to Python dictionary
    data = response.json()
        # Now you can work with the data like any Python dictionary
    print(f"Post title: {data['title']}")
    print(f"Post body: {data['body']}")

The .json() method automatically converts the response to Python dictionaries and lists so that you can look at the data using normal Python syntax from within your script.

Python Code to Fetch and Extract Data from an API

After you've made a successful API call, you'll need to extract the data you want from your response payload. Here's how.  

Saving API Data: JSON and CSV Examples

Python makes analyzing and using your data very easy with its built-in libraries.

Writing API Data to a JSON File

Saving API data as JSON lets you keep its structure, so it's perfect for when you need to have the original format to work with:

import requests
import json
# Get data from an API
response = requests.get('https://jsonplaceholder.typicode.com/posts')
data = response.json()
# Save to a JSON file with nice formatting
with open('api_data.json', 'w') as file:
    json.dump(data, file, indent=4)    
print('Data saved to api_data.json')

The indent=4 parameter makes the saved file human-readable with proper spacing and line breaks. You can go through the file and look for any inconsistencies or issues before feeding it back into another script if you want.

Saving API Data as a CSV File

CSVs generally work better for data analysis because they open natively in Excel and other tools, which lets you set up headers and columns just how you need them to be. You can also set up all this information from within the script itself if you need to:

import requests
import csv
# Get data from an API
response = requests.get('https://jsonplaceholder.typicode.com/posts')
posts = response.json()
# Save to CSV
with open('api_data.csv', 'w', newline='') as file:
    # Create a CSV writer
    writer = csv.writer(file)

    # Write header row
    writer.writerow(['id', 'title', 'body'])

    # Write data rows
    for post in posts:
        writer.writerow([post['id'], post['title'], post['body']])

print('Data saved to api_data.csv')

If your API returns nested data (layers of data within layers of data), you'll probably want to flatten it first by making it all uniform on the same level. Alternatively, you can use the pandas library, which will give you data ready for Excel straight from your script.

Handling API Errors in Python

APIs are great, but they aren't always 100% reliable. You will encounter network failures, server issues that cause them to go down, and request rate limits. Good API code usually handles these scenarios with retries, logging, and messaging.

Using Try-Except for Failed Requests

If you wrap your API calls in try-except blocks, you’ll to catch and handle errors while your script is running:

import requests
try:
    response = requests.get('https://api.example.com/data', timeout=5)
    response.raise_for_status()  # Raises an exception for 4XX/5XX responses    
    data = response.json()
    print(f"Retrieved {len(data)} records")
except requests.exceptions.HTTPError as e:
    print(f"HTTP error: {e}")
except requests.exceptions.ConnectionError:
    print("Connection error: Check your internet connection")
except requests.exceptions.Timeout:
    print("Timeout error: The server took too long to respond")
except requests.exceptions.RequestException as e:
    print(f"Error making request: {e}")
except ValueError:
    print("Error parsing JSON response")

Timeout settings prevent your code from hanging forever if the server doesn't respond in time or at all. If you are building apps that rely on external APIs for data, it's good practice to try to handle any errors.

Managing API Rate Limits

Some APIs limit how many requests you can make per minute or day. If you don’t want to get your IP address blocked or blacklisted from an API, then you will have to play by the rules that have been set out in the documentation for that API:

import requests
import time
max_retries = 3
retry_delay = 2  # seconds
for attempt in range(max_retries):
    response = requests.get('https://api.example.com/data')    
    # If we hit a rate limit (usually status code 429)
    if response.status_code == 429:
        print(f"Rate limited. Waiting {retry_delay} seconds...")
        time.sleep(retry_delay)
        retry_delay *= 2  # Exponential backoff to prevent rate limiting
    else:
        break  # Success or different error displays here

This script has a built-in system that will back off so that it gives the API time to reset the values between its attempts. This means that it won't spam the API with requests and trigger rate limiting when it's trying to receive data from an API endpoint. 

Rate limiting is an API protection that prevents APIs from being overwhelmed with requests and then crashing (something we don’t want to trigger when we have a lot of requests to get through).

Next Steps: Automating API Calls in Python

Once you're comfortable making basic API calls, you can build more advanced scripts and apps that run on schedules, or even respond to system events or messages.

Scheduling API Requests for Automatic Updates

For data that changes quite often, you can set up scheduled jobs to fetch fresh information:

import schedule
import time
import requests
def fetch_weather_data():
    """Get current weather data and save it to a file"""
    response = requests.get('https://api.weatherapi.com/v1/current.json',
                           params={'key': 'YOUR_API_KEY', 'q': 'New York'})

    if response.status_code == 200:
        data = response.json()
        temp = data['current']['temp_c']
        condition = data['current']['condition']['text']

        with open('weather_log.txt', 'a') as file:
            file.write(f"{time.strftime('%Y-%m-%d %H:%M:%S')}: {temp}°C, {condition}\n")

        print(f"Weather logged: {temp}°C, {condition}")

# Run once when the script starts
fetch_weather_data()

# Then run every hour
schedule.every(1).hour.do(fetch_weather_data)
while True:
    schedule.run_pending()
    time.sleep(60)  # Check every minute

This script logs weather conditions every hour. If weather isn’t something you want to monitor, you can change it to track more interesting things like stock prices, social media metrics, or any other changing data that you would like to monitor.

If you don’t want a script running all the time, you can use your operating system's scheduler. Cron on Linux/Mac or Task Scheduler on Windows are good options for running your Python script at set times.

Learning More with Python API Integrations

These simple code examples barely scratch the surface of what is possible with APIs. Python's API capabilities also give you the ability to use:

• Authentication to protected APIs using OAuth, API keys, or tokens

• Work with pagination (multiple pages) for bigger datasets

• Stream live data from WebSocket APIs

• Create your own API with Flask or FastAPI to serve your data to others

By mastering Python for API data extraction, you'll build a valuable skill set that can be applied across many IT domains, from data engineering to automation and DevOps. 

Ready to take your Python skills to the next level? The Python Certified Entry-Level Developer (PCEP) course will help you build the programming foundation you need to work with APIs and much more.