Github Trending Topics
This template is a simple example of how to build a data pipeline for Github trending topics.
Getting Started
Clone the repository:
To get started with this template, you can run the following command:
npx create-moose-app moose-github-dev-trends github-dev-trendsInstall dependencies:
cd moose-github-dev-trends/moose-backend && npm installProject Structure
moose-github-dev-trends/
├── live-dev-trends-dashboard/ # Frontend dashboard application
├── moose-backend/ # Backend services
└── README.md # Project documentationRun the application
npm run devStart the Workflow to Poll GitHub for Trending Topics
This workflow will run a script every 60 seconds to poll GitHub for trending topics:
moose workflow run getGithubEventsScheduled Workflows
The workflow is scheduled to run every 60 seconds. Learn how to schedule workflows.
(Bonus) Create a .env file and add your GitHub API Key
You will be able to poll the GitHub API without any authentication credentials, but you will be limited to 60 requests per hour.
To increase the limit, you can create a .env file and add your GitHub personal access token:
touch .envGITHUB_TOKEN=<your-github-api-key>DO NOT COMMIT THIS FILE TO VERSION CONTROL
Add .env to your .gitignore file:
.envCreating a GitHub Personal Access Token
Explore the Data
The template includes a dashboard application that you can use to explore the data. It is located in the live-dev-trends-dashboard directory.
cd moose-github-dev-trends/live-dev-trends-dashboard && npm install && npm run devView the Dashboard
Go to http://localhost:3000 to view the dashboard.
Explore the API Powering the Dashboard
The dashboard is powered by a Consumption API in the moose-backend project. The code is located in the app/apis/topicTimeseries.ts file.
Feel free to check out how the code works, or even modify it to suit your own interests!