Popularity
1.6
Growing
Activity
8.9
Growing
46
2
11

Description

This is a tiny web application to expose HTTP endpoints which deliver RSS feeds built by the html2rss gem.

Out of the box you'll get all configs from html2rss-configs. You can - optionally - create your own configs and keep them private.

Programming language: Ruby
License: MIT License
Tags: Rss     Feed Readers     RSSminer     Docker     Scraper    

html2rss-web alternatives and similar software solutions

Based on the "RSSminer" category.
Alternatively, view html2rss-web alternatives based on common mentions on social networks and blogs.

Do you think we are missing an alternative of html2rss-web or a related project?

Add another 'RSSminer' Software solution

README

html2rss logo

html2rss-web mergify-status

This is a small web application to deliver RSS feeds built by html2rss via HTTP.

Features:

  • serves your own feeds: set up your feed configs in a YAML file. See html2rss' README for documentation.
  • comes with all html2rss-configs included.
  • handles caching and HTTP Cache-Headers.

This web application is distributed in a rolling release fashion from the master branch.

πŸ’“ Depending on this application? Feel free to donate! Thank you!

Using the included html2rss-configs

Build the URL like this:

The feed config you'd like to use:
lib/html2rss/configs/domainname.tld/whatever.yml
β€Œ β€Œ β€Œ β€Œ β€Œ β€Œ β€Œ β€Œ β€Œ β€Œ β€Œ β€Œ β€Œ β€Œ β€Œ β€Œ β€Œ β€Œ β€Œ β€Œ β€Œ β€Œ^^^^^^^^^^^^^^^^^^^^^^^^^^^

The corresponding URL:
http://localhost:3000/domainname.tld/whatever.rss
β€Œ β€Œ β€Œ β€Œ β€Œ β€Œ β€Œ β€Œ β€Œ β€Œ β€Œ β€Œ β€Œ β€Œ β€Œ β€Œ β€Œ β€Œ β€Œ β€Œ β€Œ β€Œ ^^^^^^^^^^^^^^^^^^^^^^^^^^^

πŸ‘‰ See file list of all html2rss-configs.

Deployment with Docker

Install Docker CE and docker run -d -p 3000:3000 gilcreator/html2rss-web.

To use your private feed configs, mount a feed.yml into the /app/config/ folder.

docker run -d --name html2rss-web \
  --mount type=bind,source="/path/to/your/config/folder,target=/app/config" \
  -p 3000:3000 \
  gilcreator/html2rss-web

Automatic updating

A primitive way to automatically update your Docker instance is to set up this script as a cronjob:

#!/bin/bash
set -e
docker pull -q gilcreator/html2rss-web
(docker stop html2rss-web && docker rm html2rss-web) || :
docker run -d --name html2rss-web --restart=always -p 3000:3000  \
  --mount type=bind,source="/home/deploy/html2rss-web/config,target=/app/config" \
  gilcreator/html2rss-web

The cronjob for updating every 30 minutes could look like this:

*/30 *  * * * /home/deploy/html2rss-web/update > /dev/null 2>&1

Heroku one-click deployment

Deploy

Since this repository receives updates frequently, you'd need to update your instance yourself.

Run it locally

  1. Install Ruby >= 2.6.
  2. gem install bundler foreman
  3. bundle
  4. foreman start

html2rss-web now listens on port 5000 for requests.

Build and run with docker locally

This approach allows you to play around without installing Ruby on your machine. All you need to do is install and run the Docker daemon.

# Build image from Dockerfile and name/tag it as html2rss-web:
docker build -t html2rss-web -f Dockerfile .

# Run the image and name it html2rss-web-dev:
docker run \
  --detach \
  --mount type=bind,source=$(pwd)/config,target=/app/config \
  --name html2rss-web-dev \
  html2rss-web

# Open a interactive TTY with the shell `sh`:
docker exec -ti html2rss-web-dev sh

# Stop and cleanup container
docker stop html2rss-web-dev
docker rm html2rss-web-dev

# Remove the image
docker rmi html2rss-web

Feed configs runtime health checks

Websites often change their markup. To get notified when one of your own configs break, use the /health_check.txt endpoint.

It will respond with success if your feeds are generatable. Otherwise it will not print success, but states the broken config names.

Supported ENV variables

Name Description
PORT default: 3000
RACK_ENV default: 'development'
RACK_TIMEOUT_SERVICE_TIMEOUT default: 15
WEB_CONCURRENCY default: 2
WEB_MAX_THREADS default: 5