Mike's Wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== NewsBlur Installation on Ubuntu Server ======

This article documents the working NewsBlur installation on the Ubuntu server.

It intentionally describes the **final working configuration**, rather than every failed command or troubleshooting attempt made during the original installation.

NewsBlur is running as a collection of Docker containers. The NewsBlur application/source files live on the server's internal Linux filesystem, while the large persistent database and search-engine data directories are stored on the external SSD mounted at:

<code>
/mnt/media8
</code>

===== Installation Layout =====

The main NewsBlur working directory is:

<code>
/home/mike/docker/newsblur
</code>

or, when logged in as ''mike'':

<code>
~/docker/newsblur
</code>

The NewsBlur Docker configuration bind-mounts this directory into the applicable containers as:

<code>
/srv/newsblur
</code>

This is important because changes made to files under:

<code>
/srv/newsblur
</code>

inside the NewsBlur containers are actually changes to:

<code>
/home/mike/docker/newsblur
</code>

on the host.

For example:

<code>
/srv/newsblur/utils/feed_fetcher.py
</code>

inside ''newsblur_web'' corresponds to:

<code>
~/docker/newsblur/utils/feed_fetcher.py
</code>

on the Ubuntu host.

This also means local source-code modifications generally survive Docker container recreation.

However, a future Git update to the NewsBlur source can overwrite or conflict with locally modified files.

===== External SSD Storage =====

The NewsBlur source code remains under:

<code>
~/docker/newsblur
</code>

The persistent database/search data is stored on the external SSD mounted at:

<code>
/mnt/media8
</code>

NewsBlur-specific persistent data is kept under:

<code>
/mnt/media8/docker/newsblur/
</code>

The major directories are:

<code>
/mnt/media8/docker/newsblur/postgres
/mnt/media8/docker/newsblur/redis
/mnt/media8/docker/newsblur/mongo
/mnt/media8/docker/newsblur/elasticsearch
</code>

This keeps large and frequently changing application data off the server's main Linux filesystem.

It also means that **/mnt/media8 must be mounted before NewsBlur is expected to operate normally**.

===== Important Server-Specific Mount Policy =====

External/data drives on this server are configured so that a missing or failed external drive does **not** prevent Ubuntu from booting.

When configuring the mount in ''/etc/fstab'', use the server's normal ''nofail'' policy.

The general philosophy is:

* Ubuntu must remain bootable even if an external drive fails.
* Docker may subsequently have services that fail because their data is unavailable.
* A failed Docker service is preferable to a server that cannot boot.

Do not remove ''nofail'' from external-drive mounts without a specific reason.

===== NewsBlur Data Directory Permissions =====

During installation, the persistent NewsBlur directories required ownership matching the users used inside the respective containers.

The working ownership configuration used during setup was:

<code bash>
sudo chown -R 999:999 /mnt/media8/docker/newsblur/postgres
sudo chown -R 999:999 /mnt/media8/docker/newsblur/redis
sudo chown -R 999:999 /mnt/media8/docker/newsblur/mongo
sudo chown -R 1000:0 /mnt/media8/docker/newsblur/elasticsearch
</code>

The directories were also restricted with:

<code bash>
sudo chmod 750 /mnt/media8/docker/newsblur/postgres
sudo chmod 750 /mnt/media8/docker/newsblur/redis
sudo chmod 750 /mnt/media8/docker/newsblur/mongo
sudo chmod 750 /mnt/media8/docker/newsblur/elasticsearch
</code>

These permissions are intentionally different from the normal shared-media configuration used elsewhere on the server.

Do **not** automatically change these NewsBlur application-data directories to ''mike:media'' or ''2775'' simply because that convention is used for media storage.

The database containers expect their own UID/GID ownership.

===== Main Docker Services =====

The NewsBlur Docker Compose installation contains several cooperating services.

Important containers include:

* ''newsblur_web'' — main NewsBlur Python/Django application
* ''newsblur_celery'' — background jobs and scheduled feed fetching
* ''newsblur_node'' — NewsBlur Node service
* ''newsblur_nginx'' — internal Nginx frontend
* ''newsblur_db_postgres'' — PostgreSQL database
* ''newsblur_db_redis'' — Redis
* ''newsblur_db_mongo'' — MongoDB
* ''newsblur_db_elasticsearch'' — Elasticsearch
* ''newsblur_imageproxy'' — image proxy
* ''newsblur_mcp'' — NewsBlur MCP service
* HAProxy — internal NewsBlur proxy/load-balancing component

Most services use:

<code>
restart: unless-stopped
</code>

This allows the NewsBlur stack to return automatically after a normal server or Docker restart.

===== Important Bind Mounts =====

The main NewsBlur source tree is mounted into several containers.

For example:

<code>
/home/mike/docker/newsblur -> /srv/newsblur
</code>

This is used by at least the web application and the Celery/background-worker infrastructure.

Consequently, when modifying NewsBlur Python code, restarting only ''newsblur_web'' may not be sufficient.

If the changed code is also used for background jobs or feed fetching, restart Celery as well:

<code bash>
docker restart newsblur_web
docker restart newsblur_celery
</code>

===== Local Service Ports =====

Several NewsBlur services publish ports only to ''127.0.0.1''.

Examples from the working configuration include:

<code>
newsblur_web       127.0.0.1:8000
newsblur_nginx     127.0.0.1:8181
newsblur_node      127.0.0.1:8008
imageproxy         127.0.0.1:8088

PostgreSQL         127.0.0.1:5434
Redis              127.0.0.1:6579
MongoDB            127.0.0.1:29019
Elasticsearch      127.0.0.1:9200 / 9300 </code>

HAProxy exposes the NewsBlur HTTP/HTTPS frontend using the ports configured in the Compose stack.

Binding most internal services to ''127.0.0.1'' prevents the database and internal application services from being directly exposed to the LAN or Internet.

===== Starting NewsBlur =====

Go to the NewsBlur directory:

<code bash>
cd ~/docker/newsblur
</code>

Start or update the existing containers with:

<code bash>
docker compose start
</code>

Check their status with:

<code bash>
docker compose ps
</code>

===== Restarting NewsBlur =====

A single service can be restarted without bringing down the entire installation.

For example:

<code bash>
docker restart newsblur_web
</code>

For changes affecting background feed processing:

<code bash>
docker restart newsblur_celery
</code>

To restart both:

<code bash>
docker restart newsblur_web newsblur_celery
</code>

===== Viewing Logs =====

Standard Docker logs can be viewed with:

<code bash>
docker logs newsblur_web
</code>

or:

<code bash>
docker logs newsblur_celery
</code>

Follow a log live with:

<code bash>
docker logs -f newsblur_web
</code>

NewsBlur also maintains its own application log within the bind-mounted source tree:

<code>
~/docker/newsblur/logs/newsblur.log
</code>

This can be useful for diagnosing individual feed failures.

For example:

<code bash>
grep -i "failed" ~/docker/newsblur/logs/newsblur.log
</code>

===== Inspecting the Actual Docker Configuration =====

Because NewsBlur has many services and mounts, ''docker compose config'' is useful for seeing the final configuration after Compose processing.

From:

<code bash>
cd ~/docker/newsblur
</code>

run:

<code bash>
docker compose config
</code>

To inspect the mounts of an individual container:

<code bash>
docker inspect newsblur_web --format '{{range .Mounts}}{{println .Type "|" .Source "|" .Destination}}{{end}}'
</code>

For this installation, this confirms:

<code>
bind | /home/mike/docker/newsblur | /srv/newsblur
</code>

===== Updating NewsBlur =====

Because this installation uses the NewsBlur source repository directly, remember that local source modifications can appear in Git.

This installation currently has important local modifications to:

<code>
docker-compose.yml
utils/feed_fetcher.py
</code>

The ''docker-compose.yml'' changes are part of this server's local Docker, networking, port-binding, and storage configuration.

The ''utils/feed_fetcher.py'' change is the local YouTube RSS workaround described below.

Before updating, check for local changes:

<code bash>
cd ~/docker/newsblur
git status
git diff
</code>

If ''git diff'' opens in a pager and displays ''(END)'', press:

<code>
q
</code>

to return to the command prompt.

A ''git pull'' may refuse to proceed or create a conflict if upstream NewsBlur modifies a file that has also been modified locally.

Do **not** blindly discard local changes before determining what they are.

Next, check whether upstream NewsBlur has changed:

<code bash>
git fetch
git status
</code>

''git fetch'' checks for upstream changes without modifying the local NewsBlur files.

If Git reports:

<code>
Your branch is up to date with 'origin/main'.
</code>

there is no NewsBlur source update to install and the source-update procedure can stop here.

==== Check Local Files Against Upstream ====

Before pulling an upstream update, check whether NewsBlur has changed either of the files that are modified locally:

<code bash>
git diff HEAD..origin/main -- docker-compose.yml
git diff HEAD..origin/main -- utils/feed_fetcher.py
</code>

If either command produces output, upstream has changed that file since the currently installed version.

Review those changes before proceeding.

Even if no upstream changes are shown for these files, save copies of the local modifications before updating.

==== Save the Local Docker Configuration ====

Save the local ''docker-compose.yml'' changes as a patch:

<code bash>
git diff -- docker-compose.yml > ~/newsblur-docker-compose-local.patch
</code>

Also save a complete copy of the current local Compose file:

<code bash>
cp docker-compose.yml ~/newsblur-docker-compose.yml.backup
</code>

The complete backup is useful because the Compose file contains important server-specific settings such as local port bindings and database/storage locations.

Optionally review the saved patch:

<code bash>
cat ~/newsblur-docker-compose-local.patch
</code>

==== Check and Save the Local YouTube RSS Modification ====

This NewsBlur installation includes a local modification to:

<code>
utils/feed_fetcher.py
</code>

The modification allows direct YouTube RSS URLs such as:

<code>
https://www.youtube.com/feeds/videos.xml?channel_id=...
</code>

to bypass NewsBlur's normal YouTube API-based fetcher and instead be handled as regular RSS feeds.

Inspect the current upstream YouTube handling:

<code bash>
git show origin/main:utils/feed_fetcher.py | \
grep -n -A 8 -B 8 'is_youtube_feed_address(address)'
</code>

If upstream still contains:

<code python>
if is_youtube_feed_address(address):
</code>

and does not otherwise provide equivalent handling for direct ''/feeds/videos.xml'' URLs, the local workaround is still needed.

Save a copy of the local modification:

<code bash>
git diff -- utils/feed_fetcher.py > ~/newsblur-youtube-rss-fix.patch
</code>

Optionally review it:

<code bash>
cat ~/newsblur-youtube-rss-fix.patch
</code>

The saved patch should include the important change:

<code diff>
-        if is_youtube_feed_address(address):
+        if is_youtube_feed_address(address) and "/feeds/videos.xml" not in address:
</code>

==== Temporarily Stash the Local Changes ====

Before pulling the update, temporarily stash both locally modified files:

<code bash>
git stash push -m "Local NewsBlur server modifications" -- docker-compose.yml utils/feed_fetcher.py
git status
</code>

Confirm that neither ''docker-compose.yml'' nor ''utils/feed_fetcher.py'' is still listed as modified.

The existing untracked backup files, such as:

<code>
docker-compose.yml.original
utils/feed_fetcher.py.bak
</code>

may still appear under ''Untracked files''. This is normal and does not prevent the update.

==== Update the NewsBlur Source ====

If Git reports that the local branch is behind the upstream branch and the local modifications have been safely backed up and stashed, update the source with:

<code bash>
git pull --ff-only
</code>

''--ff-only'' prevents Git from automatically creating a merge commit if the local and upstream histories have unexpectedly diverged.

==== Review the Updated Files Before Restoring Local Changes ====

After the pull completes, inspect the updated Compose file:

<code bash>
git diff HEAD@{1}..HEAD -- docker-compose.yml
</code>

If upstream has substantially changed the Compose configuration, review those changes before restoring the old local version.

Also inspect the updated YouTube handling:

<code bash>
grep -n -A 8 -B 8 'is_youtube_feed_address(address)' utils/feed_fetcher.py
</code>

If NewsBlur still sends direct YouTube RSS URLs through its YouTube fetcher, the local YouTube workaround is still needed.

If upstream NewsBlur appears to have fixed the underlying YouTube RSS issue, **do not blindly restore the old YouTube modification**. Review and test the new upstream behavior first.

==== Restore the Local Modifications ====

If the upstream changes do not make the local modifications obsolete or incompatible, restore them with:

<code bash>
git stash pop
</code>

Then inspect the result:

<code bash>
git status
git diff
</code>

Confirm that the local ''docker-compose.yml'' settings are still correct and that the YouTube change is present:

<code diff>
-        if is_youtube_feed_address(address):
+        if is_youtube_feed_address(address) and "/feeds/videos.xml" not in address:
</code>

If ''git stash pop'' reports a conflict, **do not discard either version**. Review the conflict and reconcile the upstream changes with the local server configuration before continuing.

The separately saved backup files remain available at:

<code>
~/newsblur-docker-compose-local.patch
~/newsblur-docker-compose.yml.backup
~/newsblur-youtube-rss-fix.patch
</code>

==== Update Docker Images and Containers ====

After the NewsBlur source and local modifications are in the desired state, update the Docker images and containers:

<code bash>
docker compose pull
docker compose up -d
docker compose ps
</code>

Confirm that the expected NewsBlur containers are running normally.

Only prune unused Docker images when desired:

<code bash>
docker image prune
</code>

===== Backup Considerations =====

The NewsBlur installation is split between two important locations.

Application/configuration/source:

<code>
/home/mike/docker/newsblur
</code>

Persistent database/search data:

<code>
/mnt/media8/docker/newsblur
</code>

A complete NewsBlur backup strategy therefore needs to account for **both locations**.

Backing up only ''~/docker/newsblur'' does not back up the PostgreSQL, Redis, MongoDB, or Elasticsearch data stored on the external SSD.

Likewise, backing up only ''/mnt/media8/docker/newsblur'' does not preserve the Compose configuration and locally modified NewsBlur source tree.

===== Known Local Modification =====

As of September 2026, this NewsBlur installation contains a small local modification that allows official YouTube RSS feeds to bypass NewsBlur's YouTube Data API fetcher.

This was necessary because valid YouTube RSS feeds were returning:

<code>
YouTube fetch failed (404)
</code>

inside NewsBlur even though the same RSS URLs worked normally in a browser.

See:

[[tech:newsblur_youtube_fix|NewsBlur YouTube RSS Fix]]