Calibre, Word and MOBI: A Tale of Three Programs

(Yes, I know, MOBI is not a program, but my blog, my headlines…)

Ever since I started blogging about ebooks, I’ve cautioned people against using Microsoft Word to format their ebooks. Not because Word is a bad program and not because it’s impossible to create ebooks with it. It’s because it’s the not quite right tool. Word’s strength lies in creating print documents or pdfs.

Recently, I’ve been cautioning people to not use Calibre to convert their Word files into MOBI files in order to sell them on Amazon. Not because Calibre is a bad program and not because it’s impossible to create MOBI files with it. It’s because it’s not quite the right tool. Calibre’s strength lies in managing a person’s digital library. It was not created to convert commercial ebook files.

EPUB files are not as troublesome as MOBI files. EPUB is EPUB is EPUB, and while each device has its own special way of rendering the file to fit the platform, the differences between devices aren’t big enough for most people to notice. A single EPUB file will work pretty much the same on a Nook as it does on an iPad.

Calibre is set up for optimum use with EPUB files. If a publisher converts a Word (html) file into an EPUB file using Calibre, then what they see there is pretty close to what a Nook or iPad reader will see.

This is not true with MOBI files. The reason is Amazon. You see, EPUB devices have evolved and changed and upgraded and gone the way all technology goes, ever upward and onward. But the device makers built the newer devices around the existing ebook platform. So an EPUB ebook formatted five years ago will work pretty much the same on a new iPad as it did on a first generation Nook. Amazon went bass-ackwards. They built the new devices then tinkered and recreated entirely new ebook platforms to fit the new devices. So a MOBI file being sold on Amazon isn’t just a MOBI file. It’s also a KF/8 file and an iOS file and an AZW3 file and god knows what else is there. I don’t quite get all the technical stuff. What I do get is that the same ebook can work fine on a Kindle Fire, but go to hell on a Paperwhite and look okay on a Kindle Keyboard and turn into gibberish if an iPad user gets hold of it.

The whys and wherefores don’t matter as much as the fact that a file formatted in a program which is optimal for printing documents and then converted with a program that is at its best with EPUB files, is going to have trouble meeting the very odd demands of Kindles.

(By the way, if you are using Scrivner or InDesign to create your ebooks for sale on Amazon, you will run into the same exact problems because Amazon is constantly tweaking and fiddling with the platform(s) and updating devices and they don’t necessarily share what they’ve done with the rest of the world.)

I realize that none of what I just wrote is going to dissuade people from using Calibre to convert their Word docs into MOBI files to sell on Amazon. I know this because people are using Word because that’s the program they know and love(hate) and they need a way to convert those Word files and Calibre is the shortest distance between A and B.

So instead of wagging my finger and clucking my tongue, I did some research. Question: Is it possible to format a file in Word and convert it with Calibre and create a MOBI file good enough to sell on Amazon? (Here, I make a very clear distinction. If your Nook died and you bought a Kindle, and you want to convert all your Nook books into MOBI files you can load onto your Kindle, Calibre is a great tool. That’s personal use. You expect that the ebook might not work completely right, but that’s okay, at least you have it. You can’t ask your paying customers to accept that standard.)

What I discovered is: Yes, it is possible.

I managed to fix the worst problems I see with Calibre-converted ebooks. I managed to create ebooks that respond properly to all the user preferences in three generations of Kindles (Kindle Keyboard, Paperwhite and Fire). I almost got Calibre to build a toc.ncx (what the user sees in the Go To features on Fires and Paperwhites) the way I want it to. I think with some more tinkering and fiddling around inside the opf file, I can fix that problem. I couldn’t get the cover to display on the bookshelf in my Paperwhite, but that’s kind of a non-issue, since Amazon will handle that when the book is uploaded. (It is only a big deal if a publisher is selling direct.)

Even though the ebooks I created this way aren’t up to my standards, they will respond to user preferences and they will look fine and read fine, and thus, they are good enough for uploading to Amazon.

There is a caveat. If you format your document, save it as an html file and convert it as is with Calibre, your ebook will be broken. It will be a substandard product you should not ask people to pay for. What you have to do first and foremost is format your Word file so it works within Calibre’s parameters, and secondly, you have to fix the html coding in the Word file.

Sound scary? It is, kind of. Word’s html coding is a nightmare, full of mso odd bits that give Kindles the hiccups. The good news is, all you really need to do is remove some very specific lines of code and rearrange a few others.

Since this post is running long and I don’t even have any pretty pictures to enliven it, (plus I have a buttload of Christmas gifts to wrap) I am going to explain how I did it in my next post. It’ll have pictures. In the meantime, if any of you, Dear Readers, have figured this out and feel like sharing in the comments, feel free.

Happy New Year, Happy Day: Smashwords Now Accepts EPUB Files

It’s finally happened. Thank you, Mark Coker.

Smashwords Supports EPUB Uploads With Smashwords Direct

“One year ago in my 2011 annual year-in-review here at the Smashwords Blog, we committed to support direct EPUB uploads to the Smashwords platform in the second half of 2012.

Today we fulfilled that commitment with the launch of Smashwords Direct.

This new capability allows our authors and publishers to upload their own professionally formatted EPUB files for sale at the Smashwords store, and for distribution to the Smashwords retail distribution network….”

Read the rest at the Smashwords blog.

What does this mean? Why is this a happy day? A portent of wonderful things to come? At its heart, it means the most important thing:

Stable ebooks

I’ve spent the past year learning how to make stable ebooks. The biggest learning curve lay in figuring out how ebooks work. I’m handicapped because I’m NOT a computer savvy person. I’ve used computers for writing since the 1980s, but quite frankly I’ve used them as glorified typewriters and fancy bookkeeping ledgers with nary a thought about the inner workings or what was going on behind the scenes (behind the screen?). I had to learn a foreign language (html) and figure out who the smart people were so I could learn from them. It’s mostly been trial and error along with plenty of indulgence from some good friends who had enough faith in me to allow me to experiment on their books.

As much as I love the bells and whistles and trying this trick and figuring out that one, the most important lesson I’ve learned is this: If the ebook isn’t stable, none of the fancy stuff matters.

Is it possible for a Do-It-Yourselfer to make a stable ebook with Word? Or Scrivener? Possible, but not probable. Word processors are the wrong tools. You can follow all the directions and be meticulous, but speaking non-tech layperson to non-tech layperson: Shit happens.

A lot of that shit comes from the hardware side of the aisle. Every device maker is dreaming that his device is going to rise as Number One Preferred By Consumers Everywhere. Retailers like Amazon and Apple want their proprietary platforms to be the One Ring That Rules Them All.

With their Meatgrinder conversion program Smashwords struggled mightily to serve a lot of masters, all of them squabbling, and many not playing nice. The goal was to make it possible for anyone to self-publish and get wide distribution. The problem inherent with trying to satisfy everybody, though, is that compromises and narrowing parameters result in an overall lower quality. Ebooks had to be stripped down to the bare bones and great care had to be taken to lessen the chances that shit would happen.

It was backward and upside-down. Here we have increasingly sophisticated ereaders and tablets, full of possibilities that have barely been touched. The wrong tool (Word) makes it too dangerous to attempt exploiting the technology.

In order to reach greater heights, in order to really open up the possibilities, to look under the hood and see what these babies can really do, the ebook must be stable.

A validated EPUB file is stable. When the end user opens their ebook, no matter what the device, it will work. If the user wants to change the line spacing or the font or whatever else their device allows them to do, the ebook will oblige. It will look good on a small screen and it will look good on a big screen. If a user has multiple devices, the ebook will be stable across the devices. The ebook will continue to work even as devices are updated, improved and changed (as long as the devices continue to base them on EPUB–knocking wood here).

What does this mean for the Do-It-Yourselfer? I’m not going to lie. Building a validated EPUB file is NOT the easiest thing in the world. I have heard on good authority that the program called Sigil does a good job and is user-friendly. Having not used it myself, I do not know. Anyone who wants to discuss it, please, feel free.

By opening up Smashwords to EPUB files, my prediction for the New Year is that we’re going to start seeing a serious uptick in the overall quality of ebook formatting. Readers will demand it. They will grow increasingly dissatisfied with bland, generic looking ebooks and unhappy with ebooks that cannot be customized by their devices. We’ll start seeing innovation, too. Right now ebooks are a digital imitation of print. Face it, printed books are just about the perfect medium for conveying text. For that purpose, there’s not much room for improvement. What I’m thinking is how ebooks are different. That’s where the innovations will arise. With a stable platform, a solid foundation from which to build, ebook producers are free to innovate.

So thank you, Mark Coker and Smashwords. I predict your Smashwords Direct publishing option is going to result in benefits far above and beyond whatever it is you envisioned.