Tech Talk: A Substack Library for R Programmers
"substackR" is available at GitHub.com/posocap/substackR
This post is a quick bit of tech talk that will probably interest less than 1% of readers, so if you came here for politics instead of programming, skip ahead to the next post now. Still here? Great! You must be a fellow data nerd.
Substack's official website offers a free 'unofficial' API. They probably call it unofficial because it's so limited that they don't want to put the Substack brand name on it. Limited, but free and potentially useful, that is. They offer snippets for the endpoints in numerous languages, but not R, the statistics-heavy data scientist's language of choice, so I went ahead and created an R library to facilitate its use.
You can install the package via GitHub or CRAN. To use devtools with GitHub:
devtools::github_install("posocap/substackR")or with pak:
pak::pak("posocap/substackR")The full code is available here. The package has been submitted to CRAN and should soon be available via `install.packages(“substackR”)`.
Authentication
Before making any API calls, you need to authenticate using your Substack API key:
library(substackR)
set_substack_key("your-substack-api-key")Functions
Fetching Latest Posts
Use `get_substack_latest()` to retrieve the latest posts from your favorite Substack publication:
latest_posts <- get_substack_latest(publication_url = "posocap.substack.com",limit = 10, offset = 0)Fetching Top Posts
To get the most liked posts, use `get_substack_top()`:
top_posts <- get_substack_top("posocap.substack.com", limit = 5)Searching Posts
You can search for posts by keyword with `get_substack_search()`:
search_results <- get_substack_search(publication_url = "posocap.substack.com", query = "data science", limit = 20)Retrieving a Single Post
To fetch a specific post by its slug, use `get_substack_post()`:
single_post <- get_substack_post(publication_url = "posocap.substack.com", slug = "your-post-slug")Error Handling
`substackR` includes error handling to ensure a smooth user experience. You’ll receive informative messages for missing API keys, HTTP errors, and JSON parsing issues.
I hope you find `substackR` useful for your data analysis or automation needs. Beware of the harsh rate limits imposed by the API. They advertise 10 requests per minute, but it seems closer to 5 requests every 3 minutes. If you find it useful, please subscribe or share this blog.