Shoebill Feed

Shoebill Feed is a self-hosted, LLM-powered news aggregator. It pulls articles from RSS/Atom feeds, Reddit, email newsletters, Mastodon, Bluesky, Lemmy, GitHub, Telegram, arXiv, and arbitrary scraped web pages into a single feed, uses a local or cloud LLM to summarize and categorize everything, and learns which topics you actually care about from your own feedback — no ads, no tracking, and nothing sold to advertisers.

This site documents how to run it, how it works internally, and how to extend it.

Contents

• Introduction
  • What is Shoebill Feed?
  • Core ideas
  • What it isn’t
  • License
• Getting Started
  • Prerequisites
  • 1. Clone and configure
  • 2. Start the stack
  • 3. Log in
  • 4. Add your first categories
  • 5. Add your first source
• Configuration
  • Database & cache
  • LLM providers
  • Reddit
  • Authentication
  • Push notifications (Web Push / VAPID)
  • Notes
• Architecture
  • Services
  • Data flow
  • Backend layout (backend/app/)
  • Frontend layout (frontend/src/)
  • Key design decisions
• Sources
  • RSS / Atom
  • Reddit
  • Email (IMAP newsletters)
  • Mastodon
  • Bluesky
  • Lemmy
  • GitHub
  • arXiv
  • Telegram
  • Generic scraper
• LLM Providers
  • Choosing a provider
  • Recommended models
  • What gets sent to the LLM
  • Processing pipeline
  • Newsletter expansion
  • Anthropic Batch API
  • Embeddings
  • Transparency
• Learning & Scoring
  • Category weights
  • Keyword weights
  • Manual overrides
  • Decay
  • Feed diversity
  • Keyword clusters
• Clustering
  • First pass: title similarity (at fetch time)
  • Second pass: after LLM processing
  • What happens once items are clustered
• User Guide
  • The feed
  • Categories
  • Sources
  • Learning
  • Preferences
  • Notifications
  • Stats
  • Admin: Users
• Deployment
  • Production compose
  • Reverse proxy
  • Volumes and persistence
  • Scaling notes
  • Upgrading
• Development
  • Local setup
  • Running the tests
  • Database migrations
  • Contributing
• MCP Server
  • Setup
  • Connecting it to Claude