Ebooks have never been given the same first-class treatment in my house that other media types have. My wife and I both have Kindles and ebooks purchased through Amazon. I’ve run some of these through Calibre to strip DRM, but books typically find their way to devices via a USB drop. Whenever either of us want a book that the other already has, it ends up being a manual process to get it on-device. A recent device refresh for our e-readers got me thinking about how I could improve this process in a local-first way.

After some trial and error, I combined Calibre, Calibre-Web, and KOReader to create a system that has:

One central library for our house
OPDS interface to browse and download books directly from e-readers
Web interface to browse and read books from other devices
Book addition workflow that includes automatic DRM stripping and format conversion
Wallabag integration

Here’s how I did it:

Library Management - Calibre

Calibre is an excellent desktop application for managing ebooks. I’ve been using it for years, but being a desktop app means separate libraries on separate devices, and having to send files back and forth when we want to share books. This time, I’ve set up a single instance of Calibre on my home server using the linuxserver/calibre docker image:

  calibre:
    image: lscr.io/linuxserver/calibre
    container_name: calibre
    environment:
      - PUID={{ puid }}
      - PGID={{ pgid }}
      - TZ={{ tz }}
    volumes:
      - {{ media_path }}/books:/data/books
      - {{ storage_path }}/appspace/calibre/consume:/consume
      - {{ appdata_path }}/calibre:/config
    restart: unless-stopped
    networks:
      - proxy
    labels:
      - traefik.enable=true
      # Calibre App
      - traefik.http.services.calibre.loadbalancer.server.port=8080
      - traefik.http.services.calibre.loadbalancer.server.scheme=http
      - traefik.http.routers.calibre.entrypoints=websecure
      - traefik.http.routers.calibre.rule=Host(`calibre.{{ domain_int }}`)
      - traefik.http.routers.calibre.service=calibre
      - traefik.http.routers.calibre.tls=true
      # OPDS Server
      - traefik.http.services.opds.loadbalancer.server.port=8081
      - traefik.http.services.opds.loadbalancer.server.scheme=http
      - traefik.http.routers.opds.entrypoints=websecure
      - traefik.http.routers.opds.rule=Host(`opds.{{ domain_int }}`)
      - traefik.http.routers.opds.service=opds
      - traefik.http.routers.opds.tls=true

This container runs the Calibre desktop app in a Guacamole server, which can be accessed via browser. It works, but you definitely feel the fact that you’re using a remote desktop. This is fine for admin tasks, but I don’t plan to have this interface be part of the regular user experience. Instead, I’m going to automate everything I want Calibre to do.

Automatic Format Conversion

In Calibre you have the option to set a preferred book format and automatically convert books to that format when you add them to your library. This is really useful for maintaining compatibility with whatever e-reader(s) you have. The default format is set under Preferences -> Behavior -> Preferred output format:

To make Calibre automatically do a conversion when books are added, Preferences -> Adding books -> Adding actions -> Automatically convert added books to the preferred output format needs to be checked:

Automatically Adding Books

I’ve set Calibre to watch the mounted /consume volume and automatically consume any book files that are copied there. This is in the Automatic adding tab:

I have the /consume directory exposed on my network as a SAMBA share, so anyone can drop an ebook file there and it will be imported automatically.

When importing books in bulk via the watch folder, Calibre will often try to import the temp files that are created while the books are in the process of being copied to the directory (ex: ._book.azw3) resulting in a bunch of failed import jobs and duplicate/corrupted books being added to the library. To deal with this, I added a custom import rule under Adding actions -> Control which files are added during bulk imports that ignores any filenames that start with a period:

Stripping DRM

For ebooks purchased on Amazon and some other sellers, they’ll be locked down with DRM because of course they will be. You’ll need the DeDRM Tools Plugin to strip the DRM from these books before you can convert them. The only DRM books I have are from Amazon, so that’s what I’m setting up here, but others are supported as well (see the DeDRM FAQ for the full list). Setting it up is pretty straightforward:

Download/extract the DeDRM_tools_x.x.x.zip and copy the DeDRM_plugin.zip to somewhere that the Calibre app will be able to reach (I just used the /data/books/ directory and deleted it after).
In Calibre, go to Preference -> Plugins -> Load plugin from file, navigate to and select the DeDRM_plugin.zip.
Acknowledge that you like to live dangerously and hit Yes, and DeDRM should now appear under the File type plugin category:

With DeDRM selected, click Customize plugin to configure it. Amazon sets their DRM to a specific Kindle or app reader. All of my books were for Kindle, so under Kindle eInk ebooks I added the serial numbers for the Kindles that we have.

Now, whenever any DRM books are imported, each key in this list will be used to try decrypting it.

Calibre Content Server

Calibre has a built-in OPDS server (they call it the Content Server) which I’m going to use to serve the library to our e-readers. To enable this, Preferences -> Sharing over the net -> Main -> Run server automatically when calibre starts needs to be checked:

Remember when we called it ‘the net’? I miss that…

The Content Server can be run without requiring any user authentication, but I prefer to have it anyway. For this, check the box to Require username and password to access the Content server. Users and passwords are set under the User accounts tab (users shouldn’t need write access).

KOReader, the reader I’ll be using, has issues with the default authentication method Calibre uses. To accommodate this, under the Advanced tab, Choose the type of authentication used needs to be set to basic.

Hit apply, start/restart the content server (Connect/share icon at the top of Calibre) and we should be good to go.

New E-Reader - Kobo Clara 2E

The idea for this project came about because I was updating our e-readers. I was interested in a device that was a bit more tinker-friendly than the Kindle. I believe you can do most of this with a jailbroken Kindle, but honestly, I wanted to try something new. The price point and feature set of the Clara 2E aligned well with what I had in mind, so that’s what I went with.

KOReader

KOReader is an alternate document viewer for Kobo and other devices. It’s not an alternate firmware - it runs inside the stock Kobo software. I used the automated installation script provided by NiLuJe on the mobileread forum. You can read through the forum post for more details, but I downloaded the KOReader package and the OS-specific installer script to my computer, connected the Kobo, and ran the script - it handled the rest.

NOTE: At the time of writing, this KOReader issue about instability on the Kobo Clara 2E is actively being discussed. I haven’t experienced it myself, but it seems like there might be a model-specific issue with the Clara 2E. Everything below is specific to KOReader, and should work with any other device supported by KOReader.

Configuring KOReader

KOReader’s configuration is stored in Lua files on the device under the /.adds/koreader/ directory, which makes setup (and backing up) quick and easy if you know what you’re doing. I’ve configured my device this way, but it can also be done by hand in KOReader if you really wanted.

Adding the OPDS Catalog

KOReader has an OPDS catalog browser, and the configs for it are found in .adds/koreader/settings.reader.lua under the opds_servers list. I added the new OPDS server from Calibre to the end of this list:

    ["opds_servers"] = {
        [1] = { ... }, 
		...
        [8] = {
            ["password"] = "MySuperSecretPassword",
            ["title"] = "BurritoVoid's Calibre Library",
            ["url"] = "https://opds.my_home_domain.net/opds",
            ["username"] = "burritovoid",
        },
    },

Note the /opds at the end of the url.

Ignore the calibre_opds and calibre_wireless_url items that are further up in this file. calibre_opds only works for local connections that don’t use a reverse proxy, and calibre_wireless_url is for the Calibre Wireless Connection which is a different feature altogether.

Adding a Dictionary

I went to public school, so from time to time I need to look words up. KOReader doesn’t include any dictionaries by default, but adding one is pretty easy. I grabbed a copy of the Oxford Dictionary 2nd Edition, which comes in 2 archives. The contents of the archives need to be extracted and placed in the .adds/koreader/data/dict directory. Any dictionaries dropped here will automatically be used for word lookup when reading.

Wallabag Integration

Most of my news/article reading goes through a self-hosted Wallabag server, and KOReader integrates with it! Client configuration requires the Client ID, Client Secret, Username, and Password from the Wallabag server. The config file is at .adds/koreader/settings/wallabag.lua:

return {
    ["wallabag"] = {
        ["articles_per_sync"] = 30,
        ["auto_tags"] = "",
        ["client_id"] = "YOUR_CLIENT_ID_HERE",
        ["client_secret"] = "YOUR_CLIENT_SECRET_HERE",
        ["directory"] = "/mnt/onboard/wallabag/",
        ["download_queue"] = {},
        ["filter_tag"] = "",
        ["ignore_tags"] = "",
        ["is_archiving_deleted"] = true,
        ["is_auto_delete"] = false,
        ["is_delete_finished"] = true,
        ["is_delete_read"] = false,
        ["is_sync_remote_delete"] = false,
        ["password"] = "YOUR_WALLABAG_PASSWORD_HERE",
        ["remove_finished_from_history"] = false,
        ["send_review_as_tags"] = false,
        ["server_url"] = "https://WALLABAG.YOURSERVER.COM",
        ["username"] = "YOUR_WALLABAG_USERNAME_HERE",
    },
}

I also set the is_archiving_deleted and is_delete_finished to true. These settings are poorly named…

is_delete_finished will mark articles that you finish reading for deletion from the server.
is_archiving_deleted means that items marked for deletion will actually just be archived on the server instead of deleted.

Simply put, when I mark an article as finished in KOReader, it’ll archive it on the server.

Starting KOReader

That’s it for the device configs. After ejecting the Kobo and booting into KOReader (via the newly added NickelMenu menu item), the OPDS catalog is found under the magnifying glass icon at the top of the main menu. From here, you can browse and download the entire library.

The OPDS catalog is the only screen in KOReader I couldn’t find a way to get a screenshot of…

Browsing for Books - Calibre-Web

Making the entire library browsable and accessible using only the Kobo was key for simplicity, but it’s not the most elegant way to browse the library, especially a large one. To address this, I also set up a Calibre-Web server using the linuxserver/calibre-web image:

  calibre-web:
    image: lscr.io/linuxserver/calibre-web
    container_name: calibre-web
    environment:
      - PUID={{ puid }}
      - PGID={{ pgid }}
      - TZ={{ tz }}
      - DOCKER_MODS=linuxserver/mods:universal-calibre #optional
      - OAUTHLIB_RELAX_TOKEN_SCOPE=1 #optional
    volumes:
      - {{ appdata_path }}/calibre-web:/config
      - {{ media_path }}/books:/books:ro
    networks:
      - proxy
    labels:
      - traefik.enable=true
      - traefik.http.routers.calibre-web.entrypoints=websecure
      - traefik.http.routers.calibre-web.rule=Host(`books.{{ domain_int }}`)
      - traefik.http.routers.calibre-web.tls=true
    restart: unless-stopped

Setup for this is pretty simple, you just need to provide Calibre-web with the location of your Calibre library. What you end up with is a sleek catalog of your library, from which you can browse and read.

End

We’ve been running this setup in our house for about a month now, and it’s done everything we’ve wanted it to do. My wife finds it easy to use, and I get the full control over our library and it’s distribution that I wanted. I also got 2 old Kindles to hack away at.

Centralized Ebook Management with Calibre, Calibre-Web, and KOReader