How I self-host my ebooks and audiobooks
4 min read
This is the companion to I own my books, where I made the case for stripping DRM and actually owning what I buy. This post is the machinery overview: how I store, serve, and read all of it from my own homelab.
My books workflow #
When I find a book I want to buy, I go to eBooks.com and search for it. They have most titles, but occasionally I will have to go to Kobo to find it; regardless, I purchase the book.
Note: This is just an overview of how this works and is not a technical, step-by-step walk-thru.
Calibre #
If the book has DRM, the distributor gives me a file with the .acsm extension. If the book is DRM free, then they hand me the ePUB file directly. Either way, the next step is to import the downloaded file into Calibre. With the right plug-ins, Calibre will automatically download the book, remove the DRM (if any) and add the ePUB file to the Calibre books library.
Syncthing #
Once the book is in Calibre as a DRM-free ePUB, Syncthing syncs it to one of my homelab instances that serves it with Calibre-Web.
Calibre-Web #
I use Calibre-Web to then serve my Calibre books on my local network. Calibre-Web implements the same endpoints the Kobo store exposes: sync, library metadata, cover images, reading state. This is the first piece that allows me to connect my books library to my Kobo device.
DNS and certificates #
Two things have to be in place to allow my Kobo to connect to Calibre-Web. My local DNS has to point an internal hostname at the server running Calibre-Web. And that server needs a real, publicly trusted TLS certificate, because the Kobo thinks it's talking to a commercial store and won't accept a self-signed one. Getting a legitimate certificate for a hostname that only exists inside my homelab is the fiddly part. I may write that up separately.
Kobo #
Now I update the api_endpoint setting in the .kobo/Kobo/Kobo eReader.conf file on the Kobo itself so it points to the internal hostname from the DNS and certificates section above. I plug the Kobo into my computer over USB and it mounts as a drive, which is how I get at the file. There's no way to do this from the device's settings menus.
All together now #
So my workflow looks like this in practice:
- Buy and download the book.
- Import the book into Calibre.
- Open my Kobo device and download the new book to the reader.
- Read.
The default software on the Kobo is a first-class reading experience. Connecting it to my Calibre-Web server keeps that experience intact while my whole library syncs down over my local network.
KOReader #
I see many people talking about KOReader, so when I first started on this path, I had every intention of installing and using it. In my own tests, the software just felt a lot rougher and harder to make into a nice reading experience, and I still had to connect my device directly to my laptop to load books. No shade towards KOReader and all the people who seem to love it, but the Kobo reading software works perfectly for me, and I can't think of a feature that KOReader provides that isn't already in the Kobo software.
Audiobooks #
I listen to a fair amount of audiobooks, and my preference is to purchase them from Libro.fm. Their audiobooks are already DRM-free, so I just download the files from the Libro.fm website and boom, I have my backup.
Audible books do have DRM, but I can remove it. In order to make my backups, I use ffmpeg with credentials that audible-cli legitimately pulls from my authenticated account. I'm decrypting content I bought, using a key I'm entitled to, into a universally playable format (.m4b). So I can back up even Audible exclusives like Dungeon Crawler #8.
No matter how I purchased and backed up the audiobooks, I prefer to listen to them in Audiobookshelf, which is a self-hosted solution in my homelab.
Conclusion #
And that's my books workflow and an overview of how I self-host all this. I'm really interested in hearing about others' book workflows too. Drop me a line if you've written something up and want to share.
curbstickle via Lemmy
I don't do DRM, so the majority of my workflow is simplified. Download to the correct directory, auto-imported by Calibre and flagged for review of metadata, manually review metadata, convert to epub3 (even if its v3 to v3, I've removed wonky behavior before by doing so), export to appropriate directory for the OPDS server.
Basically the same for audiobooks, but dropped into the directory for AudioBookShelf.
Australis13 via Lemmy
Thanks for sharing. I would definitely be interested in a write-up on how you solved the DNS+certificate problem please, as this is a problem I've run into on my self-hosting journey.
lepinkainen via Lemmy
Need to look into your ebook flow.
I… dislike Calibre because of the “unique” UI setup and the fact that it’s next to impossible to self-host on a NAS.
But sadly it has every feature ever, which makes it essential and irreplaceable.
tburkhol via Lemmy
Just what I was looking for. Recently got a kobo for experimenting, and just starting to figure out how to integrate it.
I'd still rather physical books, but sometimes there's just too much premium for too shitty of POD trash.
Deebster via Lemmy
I'm not OP, but I use Caddy as a reverse proxy, which generates a wildcard TLS certificate via their Cloudflare integration (who host my DNS). At some point I'll move off Cloudflare but I never intend to host my own DNS (or email).
Bryan L. Fordham via Mastodon
@michaelharley Thanks for this. I stopped using Audible a while ago, but I still have a number of books I purchased hanging out there. I wasn't aware of a CLI for it, will definitely be looking into that soon!
cannedtuna via Lemmy
Good write up. This is something I’ve been putting off because of how irritating Calibre is to setup.
For Audible audiobooks, I’ve been using Liberate, which I’ve found super simple to use. When it pulls the books it strips DRM automatically. It also allows for downloading any attached PDF files for audiobooks with illustrations like Sanderson’s novels tend to have.
gedaliyah via Lemmy
Excellent writeup! Thanks for posting. Lately, I've been using Calibre Web Automated, an active fork with a lot of added features and compatibility features.
Depending on someone's setup it may or not be worth it to switch (Calibre-web works great), but I'd definitely recommend it for someone starting up.
adeoxymus via Lemmy
Note for koreader you can hook it up to kavita or audiobookshelf to download books wirelessly. I’m mainly annoyed by the manual calibre step. I have to do the same thing to remove drm, any idea if it can be automated?
rnercle via Lemmy
that's me.
my partner asked me to self-host their e-books. For now i only have jellyfin (on Debian) running to stream music locally. I bookmarked calibre-web last week as an e-book solution.
What made you choose c-w automated over c-w? Any other advices?
Good_Slate via Lemmy
I self hosted it within a docker container on Open media vault. So it is possible to host it on an nas (if you can use docker).
My ebook flow works a similar way to op's. Though I have to be on my home WiFi to download books using the kobo. Or connecting to a router with a Vpn to my server.
exu via Lemmy
For anyone who hasn't yet, do make sure to go through the default conversion settings in Calibre and turn off all the resizing. Title image resizing is a separate option turn that off as well.
If you use Koreader or modern enough devices it's probably fine to default to epub3 instead of epub2 as well.
lepinkainen via Lemmy
Yea, I have the VNC virtual docker hack running and it’s annoying. The weird UI is even worse because the virtual desktop doesn’t map properly and dialogs become full screen affairs etc.
I’m trying to use it as little as possible 😅
mesa via Lemmy
Ngl i just have an old ass file server and drag/drop things there. Then copy/paste the ebook/audiobook over to the offending device. Boom done.
Good_Slate via Lemmy
Oh yeah, I also use it as little as possible! it's far from ideal.
Wojtek Powiertowski via Mastodon
@michaelharley Thanks for the write-up, it is motivating me to do something about my Apple Books collection and go DRM-free and self-host something similar
B0rax via Lemmy
I didn’t know about libro.fm, I will check it out, thanks! It looks like it is international and should also work for German audiobooks?
gedaliyah via Lemmy
Sure. As a fork, it has all of the benefits of Calibre-web. It's stable, well established, versitile, works well with Calibre desktop software (they are three different projects, despite the similar names). Yet it has a number of different features like automatic ingest, writing/converting files, sending to different devices, etc. There is a pretty complete list on the GitHub page I linked. It has a big community and active development, so its likely that any features you might look for could be coming soon.
There are other self hosted software projects for ebooks (you can even use Jellyfin!). I had switched to Booklore at one point for a more modern UI and tracking, but it was more flash than execution and ultimately looks like it flamed out over some community issues. There are also projects like Kavita that are designed for features around comics/manga, but also do an excellent job with ebooks in general.
You should consider two things: what the library looks like, and where it will be read. If it is mostly books with a few other media (like comics, PDFs, etc.), then CWA is probably a good bet. If it's mostly comics and manga with some books and other media, consider whether Kavita is the better fit. If it is mostly DRM content, and you or if you want to manage it through Calibre Desktop, how does it connect? How often are new books being added and from where?
What screen will it ultimately be read on? If it's an ereader, check out the "books" and "self-hosted" communities here on Lemmy for which protocol, etc. is used. If it will be read on a computer or mobile device, what apps are available? Will it be read on multiple devices?
For example, if it is a small library, read on an Android/iOS device, Jellyfin with JellyBook may be fine. If it is a large manga library already tracked on AniList, Kavita may be a good bet. If features like notes and highlights are important to sync across devices, something like Anx Reader with WebDAV might even be better.
What OP describes is an excellent setup for 90% of cases.
TrickDacy via Lemmy
I was about to say, that sounds just like libation! Turns out it is the same. Either "liberate" was a typo (no shade, just noticing) or they changed the name?
In any case that app is fuckin awesome. It makes short work of freeing your entire audible library of drm.