Learning and Applying Basic API Tasks in Chronicling America

Learning and Applying Basic API Tasks in Chronicling America#

This notebook is meant to teach you some of the basic API tasks you’ll encounter using the loc.gov API.

Feel free to download this notebook and run with your own search query.

The first thing you want to do is import modules. Modules contain Python definitions and statements that you will need in order for the scripts to run.

Importing Modules#

The following modules are some of the most common you will be needing.

To import the modules you need for this notebook, simply run the following code below.

import requests
import pandas as pd
import os
import pprint
import re

Define and perform a query#

After importing modules, you will typically define your query by pasting your API Query URL.

  • In more complicated queries, you would also run functions which can refine your search.

  • Your results will need to be read in JSON format so make sure “&fo=json” is at the end of your API Query URL.

  1. Paste your Search Query URL into the locgov_url_search = '{URL}'

  2. Make sure the search query URL has &fo=json at the end

  3. When ready, Run the code.

# Define your query URL
locgov_url_search = 'https://www.loc.gov/collections/chronicling-america/?dl=page&end_date=1924-12-31&ops=PHRASE&qs=clara+bow&searchType=advanced&start_date=1924-10-01&location_state=california&fo=json'

# Run the query using the API
api_query = requests.get(locgov_url_search)

# Tell Python to read the results as json
search_result = api_query.json()

Pagination via API Query#

Other types of useful metadata includes pagination. Pagination can be used to display the # of pages of a Search Query or the current page of a newspaper image.

  • The pagination results here should be the same as those shown after you performed an Advanced Search on Chronicling America.

  • This is useful for validating the search query and checking whether there are too many results for your search. If there are too many results for your search, you may encounter issues processing your data.

  • Note: We recommend keeping search results under 100,000 hits. Please use facets to limit the size of your results and search query or you may be automatically blocked. See Limitations and Rate Limits for more information.

  1. Simply run the codes below.

  • The first will give you the pagination metadata of the search result.

search_result['pagination']

{'current': 1,
 'first': None,
 'from': 1,
 'last': None,
 'next': None,
 'of': 2,
 'page_list': [{'number': 1, 'url': None}],
 'perpage': 40,
 'perpage_options': [20, 40, 80, 160],
 'previous': None,
 'results': '1 - 2',
 'to': 2,
 'total': 1}

  • The second will print out the pagination metadata with more context.

print('Current page:')
print(search_result['pagination']['current'])

print('\nPath to request the next page:')
print(search_result['pagination']['next'])

print('\nTotal number of results:')
print(search_result['pagination']['of'])

print('\nTotal number of results per page:')
print(search_result['pagination']['perpage'] )

print('\nTotal number of pages:')
print(search_result['pagination']['total'])

Current page:
1

Path to request the next page:
None

Total number of results:
2

Total number of results per page:
40

Total number of pages:
1