Calibre, Word and MOBI: A Tale of Three Programs

(Yes, I know, MOBI is not a program, but my blog, my headlines…)

Ever since I started blogging about ebooks, I’ve cautioned people against using Microsoft Word to format their ebooks. Not because Word is a bad program and not because it’s impossible to create ebooks with it. It’s because it’s the not quite right tool. Word’s strength lies in creating print documents or pdfs.

Recently, I’ve been cautioning people to not use Calibre to convert their Word files into MOBI files in order to sell them on Amazon. Not because Calibre is a bad program and not because it’s impossible to create MOBI files with it. It’s because it’s not quite the right tool. Calibre’s strength lies in managing a person’s digital library. It was not created to convert commercial ebook files.

EPUB files are not as troublesome as MOBI files. EPUB is EPUB is EPUB, and while each device has its own special way of rendering the file to fit the platform, the differences between devices aren’t big enough for most people to notice. A single EPUB file will work pretty much the same on a Nook as it does on an iPad.

Calibre is set up for optimum use with EPUB files. If a publisher converts a Word (html) file into an EPUB file using Calibre, then what they see there is pretty close to what a Nook or iPad reader will see.

This is not true with MOBI files. The reason is Amazon. You see, EPUB devices have evolved and changed and upgraded and gone the way all technology goes, ever upward and onward. But the device makers built the newer devices around the existing ebook platform. So an EPUB ebook formatted five years ago will work pretty much the same on a new iPad as it did on a first generation Nook. Amazon went bass-ackwards. They built the new devices then tinkered and recreated entirely new ebook platforms to fit the new devices. So a MOBI file being sold on Amazon isn’t just a MOBI file. It’s also a KF/8 file and an iOS file and an AZW3 file and god knows what else is there. I don’t quite get all the technical stuff. What I do get is that the same ebook can work fine on a Kindle Fire, but go to hell on a Paperwhite and look okay on a Kindle Keyboard and turn into gibberish if an iPad user gets hold of it.

The whys and wherefores don’t matter as much as the fact that a file formatted in a program which is optimal for printing documents and then converted with a program that is at its best with EPUB files, is going to have trouble meeting the very odd demands of Kindles.

(By the way, if you are using Scrivner or InDesign to create your ebooks for sale on Amazon, you will run into the same exact problems because Amazon is constantly tweaking and fiddling with the platform(s) and updating devices and they don’t necessarily share what they’ve done with the rest of the world.)

I realize that none of what I just wrote is going to dissuade people from using Calibre to convert their Word docs into MOBI files to sell on Amazon. I know this because people are using Word because that’s the program they know and love(hate) and they need a way to convert those Word files and Calibre is the shortest distance between A and B.

So instead of wagging my finger and clucking my tongue, I did some research. Question: Is it possible to format a file in Word and convert it with Calibre and create a MOBI file good enough to sell on Amazon? (Here, I make a very clear distinction. If your Nook died and you bought a Kindle, and you want to convert all your Nook books into MOBI files you can load onto your Kindle, Calibre is a great tool. That’s personal use. You expect that the ebook might not work completely right, but that’s okay, at least you have it. You can’t ask your paying customers to accept that standard.)

What I discovered is: Yes, it is possible.

I managed to fix the worst problems I see with Calibre-converted ebooks. I managed to create ebooks that respond properly to all the user preferences in three generations of Kindles (Kindle Keyboard, Paperwhite and Fire). I almost got Calibre to build a toc.ncx (what the user sees in the Go To features on Fires and Paperwhites) the way I want it to. I think with some more tinkering and fiddling around inside the opf file, I can fix that problem. I couldn’t get the cover to display on the bookshelf in my Paperwhite, but that’s kind of a non-issue, since Amazon will handle that when the book is uploaded. (It is only a big deal if a publisher is selling direct.)

Even though the ebooks I created this way aren’t up to my standards, they will respond to user preferences and they will look fine and read fine, and thus, they are good enough for uploading to Amazon.

There is a caveat. If you format your document, save it as an html file and convert it as is with Calibre, your ebook will be broken. It will be a substandard product you should not ask people to pay for. What you have to do first and foremost is format your Word file so it works within Calibre’s parameters, and secondly, you have to fix the html coding in the Word file.

Sound scary? It is, kind of. Word’s html coding is a nightmare, full of mso odd bits that give Kindles the hiccups. The good news is, all you really need to do is remove some very specific lines of code and rearrange a few others.

Since this post is running long and I don’t even have any pretty pictures to enliven it, (plus I have a buttload of Christmas gifts to wrap) I am going to explain how I did it in my next post. It’ll have pictures. In the meantime, if any of you, Dear Readers, have figured this out and feel like sharing in the comments, feel free.

I Don’t Hate Calibre, But…

duck coverJudging by the heated storm I roused the last time I criticized Calibre it’s probably a mistake to do so again. What the hell. I have to say it:

DO NOT USE CALIBRE TO CONVERT YOUR EBOOKS FOR THE PURPOSES OF SELLING THEM ON AMAZON.

Yesterday I did a troubleshoot and repair on a writer’s ebook. The EPUB she had converted from Word via Calibre was perfectly fine. It was a valid EPUB, and it displayed the way it should on both Calibre and Adobe Digital Editions. Problems arose when she converted the EPUB into AZW3 and MOBI. The ebook worked when she loaded it onto her Paperwhite Kindle, though it had some disturbing issues. Amazon rejected the file outright.

I took her EPUB and ran it through the Kindle Previewer to see what the problem was. It converted with WARNINGS (never want to see that). The ebook opened, but the button under Devices for Kindle for iOS was greyed out. And the cover didn’t display. When I loaded it on my Paperwhite, some of the user controls were locked. I then went back and looked inside the EPUB. The Calibre conversion had declared font-families–Georgia and Times New Roman–neither of which display on Kindles, but they aren’t ignored either, and hence cause all sorts of interesting little problems. Plus, it had built a cover page.

I’m not a techie person, so I don’t know if I can explain it adequately, but I will try. When a file is uploaded to Amazon it converts the source file into a 3-in-1 ebook. If you’ll compare the size of an EPUB to a Kindle, you’ll notice that the MOBI file is about 3 times bigger than the EPUB. The ebook you buy from Amazon will work across several types of devices: e-ink readers, Fire tablets, and a variety of apps, including iOS for Apple. This is tricky stuff. While Calibre can and does convert your files into the MOBI format or AZW3 format, and the files are suitable for personal use, they aren’t suitable for commercial use because the probability is about 97.3% that an ebook converted through Calibre and then uploaded to Amazon will be broken.

This is all about Amazon. They have proprietary platforms and they are constantly updating and improving and tinkering and it’s difficult for outsiders to keep up. Calibre is a library management tool, not a commercial conversion tool. They can’t be expected to stay ahead of the Amazon updates and bugs. That’s not their purpose.

The fix for the writer’s problem ebook was fairly simple. I removed the font-family declarations, removed the cover page, added a line of code to the metadata in the content.opf so Amazon knew the cover was in the file, resized the cover and ran it through the Kindle Previewer. Voila! Everything worked the way it should. The writer was able to upload the file to Amazon with no problems.

My recommendation to the writer for future projects was this: continue using Calibre to convert her Word files to EPUB files since there appear to be no problems and her ebooks are working fine. But, then to refresh her knowledge about using stylesheets in Word via Mark Coker’s Smashwords Style Guide, and direct load her Word file to Amazon. It’s a bit of a pain since you can’t examine and adjust the file before you start the uploading process, but you can take full advantage of the previewer at Amazon and make sure everything works before you hit the Publish button. That way there won’t be any extraneous items like cover pages and “embedded” fonts to muck up the works.

So seriously, folks, don’t kill the messenger or throw rotten tomatoes. This is just the reality. Calibre is not the right tool for converting files to sell on Amazon. Use the program for what it’s intended–managing your ebook library–and find other means to deal with Amazon.

Guest Post: Karen Myers–No Need For Calibre

When I ran the recent ebook formatting contest, I noticed some similar issues with many of the entries, all of them the result of Calibre conversion. I have no beef with Calibre, it’s a great online reading platform and I use it myself, but it’s a not-quite-right tool for converting Kindle books. One entrant, Karen Myers (Perkunus Press), contacted me about the problems in her ebook.

KMblog6The flaws weren’t fatal, but Karen was not satisfied with not-quite-right. So I pointed her at Sigil and Paul Salvette’s books, tutorials and website, and now I’ll let her tell you the results:

__________________________

Silly me. I’m an old programmer and I pride myself on trying to get my ebooks “just so”, as if I were writing a piece of code. I want to create worthy offerings to add to humanity’s river of books; at the very least, they should be shiny and well-scrubbed.

So when Jaye offered to judge the formatting of a few books from her blog fans, I hopped right on board, and she was very kind in her review.  But I read with horror things like “squishy line spacing” and links to chapters not working quite as they should, systematically.

I use an EPUB reader and hadn’t seen the book on a Kindle device other than the PC Previewer, so it was useful to see this from the Kindle reader’s perspective, since none of my buyers had complained (yet). Without a Kindle device, I hadn’t realized quite how irritating it was to not properly trigger the “Cover” and “TOC” hard buttons.

Now on the one hand, it wasn’t really broken, but on the other hand, I want perfection in book formatting, and some cosmetic and graphic flourishes. I’m not willing to settle for “good enough,” so Jaye was nice enough to coach me through some of the issues.

(Jaye here: I made a template for Karen based on Paul Salvette’s The Ebook Design and Development Guide)

If you’re content with auto-conversion from EPUB to MOBI or vice versa, or output direct to ebook formats from products like Scrivener, then this is overkill for you and you can stop reading now. But if you want as much control as possible over the results without killing yourself, you might find this approach useful.

Originally I formatted my ebooks in raw HTML using tips from a variety of online recommendations, like Guido Henkel’s. The first book was a real learning curve for me, but after that it wasn’t difficult to just do the same for later books.  Automated search/replace macros took care of things like wrapping lines with <p></p> tags, and so forth.

I took the HTML output, opened it in Calibre, added the cover and converted it to EPUB using Calibre defaults (more or less). I did the same with a separate conversion to MOBI, which required me to maintain a different HTML file because of the way Calibre generates the MOBI TOC. These outputs were what I was uploading to the distributors. The MOBI files produced this way were not ideal, possibly because of AZW vs MOBI choices (in other words, me as a Calibre user, not necessarily Calibre as a tool), and I was left with two files to maintain (EPUB and MOBI) and the Smashwords EPUB as a third file, since my approach wasn’t modular.  So every time I found a typo…

Jaye has taught me a better way… (Sigil)

KM blog1KMblog2I poured my HTML file for a book, broken up into chapters, into a Sigil template.  Each part of my book has a separate file: Beginning Blurb, Title Page, Copyright Page, Also-By-This-Author Page, Small TOC, Chap 1… Chap Last, Guide & Name Index, If-You-Liked-This-Book Page, Excerpt-From-Next-Book Page, Author Bio Page, Long TOC.

KMblog3I have 3 outputs: MOBI, EPUB, Smashwords EPUB. The difference between Mobi and the two EPUBs is that the “stylesheet.css” file is a little different between MOBI and EPUB, and the Cover page is treated differently (EPUBs require an extra step). The difference between my EPUB and my Smashwords EPUB version is that the Copyright Page has different content.

Now, thinking in the long term, I expect that the differences between MOBI output and EPUB output are likely to be persistent, and other devices may come along and generate different optimum stylesheet requirements. So I’m fine with having two different (but very similar) stylesheets which I maintain externally and copy in as needed into the stylesheet.css shell.

Likewise, the fact that Smashwords requires its own ISBN means only that I maintain two different external Copyright Page HTML documents and copy the contents of whichever one I want into the shell in Sigil.

Both of these make use of a simple modular structure.

So, what happens when I finish a new book and want to format it?

PREP

  1. I copy the Sigil file (MOBI version) from the previous book and rename it.
  2. I create Copyright.HTML files for both the normal and Smashwords copyright pages by copying the ones for the last book, renaming them, and updating the content.
  3. I create a new Title page (it’s a graphic) and a new Cover.

KMblog4MAIN CONTENT (MOBI)

  1. I work on the MOBI version first (it’s the master). I copy the text in, chapter by chapter, the front blurb, and the back excerpt.  I run saved searches to wrap the lines with <p></p> tags and to convert special characters to named entities.
  2. I update the Also By and If You Like This book pages by hand.
  3. I run a saved search to update the Title field on all the HTML pages to the new work, and update the equivalent fields in the TOC.NCX and CONTENT.OPF files.
  4. If the book is a little longer or shorter (number of chapters) than the last one, I update the TOC.NCX and CONTENT.OPF files and the HTMLTOC file.
  5. I update the metadata in the TOC.NCX and CONTENT.OPF files. This allows me to do some things that either Calibre doesn’t, or I don’t know how to find, such as set a UUID (Unique User ID) for my short stories that don’t have ISBNs, embed book descriptions, add keywords, etc.  There’s a great tool for this that Jaye told me about:

bb ebooks meta-pad generator (courtesy of Paul Salvette, BB Ebooks Thailand)

KMblog5Run the file through Kindle Previewer (which runs Kindlegen) and check the results.

How much time does this take?  I just updated my entire backlist (3 novels, 5 short stories, 1 story collection) to Sigil–it takes me about an hour for a novel, and 20 minutes for a short story.  Making the short story collection from the already-formatted short story files was truly trivial.

EPUB VARIANT

  1. Substitute the content of the EPUB stylesheet into the stylesheet.css.
  2. Run the Sigil tool to insert the cover.  (Kindlegen does that a different way for MOBI).
  3. For the Smashwords variant, substitute the contents of the Smashwords copyright.html for the default one.

That’s it.  I import the files into Calibre for one last look to make sure they seem healthy, and do a quick scroll through on my EPUB reader. If I find typos, I fix them in the MOBI version (and the Scrivener original) and redo steps (10)-(12).

Why not use Calibre?  I am confused by the various options and clearly, for MOBI conversion, I wasn’t doing it quite right. Also, my original HTML file was one big file with a stylesheet and all chapters together, making modular changes clumsy. Calibre created its own version of the styles it found, and they weren’t always what I expected. It’s a big black box to me, and there were some issues with the results, which may be my fault, not Calibre’s.

Why not use Scrivener?  Like Calibre, you are at the mercy of whatever Scrivener decides to do to instantiate the different conversions. Since the Scrivener text isn’t in HTML, there are all the issues of named entity conversions to deal with, and you have little control over the default styles. The results may be clean, but you can’t do anything special, such as use graphic chapter heads, scene dividers, and so forth, at least not in the Windows version.  Perhaps there’s a way…

Other tools, like Scrivener, will take your word processor input and generate EPUB and MOBI output, but the black box in between what you write and what they produce leaves you at the mercy of the limitations of others, and so your output will remain at best functional vanilla. That’s not a bad thing, but we can do better.

It’s really not that hard to go through the learning curve once.  After that, each new book becomes quite easy.  Your book designers or people like Jaye can help you get started by setting up the first one and explaining how it works.

_____________________________

Jaye again. Thank you, Karen. Now I’d like to add a word about Sigil.

Sigil creates EPUB files. Kindle ebooks are MOBI files converted from EPUB files. So, you can take an EPUB file created in Sigil and convert it with KindleGen into a MOBI file that will work on Kindles.

There’s a gotcha–One size does NOT fit all. EPUB and MOBI handle styling differently. They handle covers differently. If you know what the differences are, you can use Sigil to create ebooks that work exactly how they are supposed to, on EPUB and Mobi. You can control how the devices handle your work, as opposed to being at the mercy of whatever the conversion program decides is best.

I have a special offer for readers of this blog. If you are really serious about creating ebooks that work properly across devices, and you are ready to climb the next step on the learning curve, go buy Paul Salvette’s The Ebook Design and Development Guide. It might freak you out the way it freaked me out when I first read it (I have zero programming experience). If it does, but you’re still motivated, send me an email (jayewmanus at gmail dot com) with a screenshot of your receipt for Paul’s book and I’ll set you up with a template similar to the one I made for Karen.

 

 

Changing Ereader Landscapes: What Can It Mean?

I don’t usually post unless I have reached a conclusion of some sort–even if it is wrong (I trust my readers to set me straight). Now I’m just puzzled. This started when I was trying to figure out yet another bug in my Kindle Keyboard. For those who don’t know, that’s the older Kindle model with the tiny keyboard on the casing. Sturdy, reliable, GREAT battery life, and here recently, full of bugs. This time it affects bold face type. Sometimes it displays, sometimes it doesn’t. And I found an instance where it partially displays.

That same day Jon Westcot sent me an email about Nook ereaders. I couldn’t suss out what was going on, though was able, as usual, to come up with some wild-ass conspiracy theories. Here is what Jon sent:

Here’s an interesting situation (meant in the truest Chinese curse way):

With most of your journal postings, you talk about formatting issues, and most of the conflicts come from the support of the “standard” that the various device developers produce and maintain.

But I have found yet another layer to this mess that I wanted to tell you about. I am currently working towards freeing my Nook Tablet from the restrictive yoke of Barnes and Noble’s operating system and replacing it with a basically fully implemented version of the Android OS. (It’s a long, geeky process, but it’s been informative, frustrating… and even fun. And yes, I know I have a “unique” idea of fun.) As part of that process, I’m trying out various ePub readers.

Oh my freaking God!

What a mess! I first downloaded the Nook for Android application. It can’t even find my library of books because I choose to keep them on the secondary memory card. I can’t even contact B&N support because what I’m doing with my Nook violates its warranty, even though the question has to do with their non-device software.

So, I turned to the Google Apps Store to look for ePub readers. The first one I grabbed, while having a huge number of downloads and a nearly 5-star rating, decided to display everything with a Chinese font!

The next one was okay — at least, it showed readable text! But the formatting was all wrong. It took a lot of wrangling to finally find the setting that lets the user accept the publisher’s formatting defaults! Preposterous!

So, I’m still looking for a good e-reader. Until then, I won’t fully migrate my Nook Tablet to the Android OS. But this whole process really surprised — and disheartened — me. It just never occurred to me that the independently-developed e-readers would be so… crappy. I would have expected just the opposite.

I guess this means that we have more things to worry about than “just” the way device developers create e-book reading software — now we have to worry about the software developers, too.

To which my mind leapt to one truly dark scenario and one big conspiracy theory.

I’ve suspected all along that much of what goes into programming and creating platforms for ereaders is created by people who have little concern–or perhaps little awareness–of end users. Is there any other industry that works this way? Maybe high fashion dress designers.

The industry WANTS us all to move happily into the land of tablets. Tablets are cool, but for reading, the dedicated ereaders are far superior. From the industry point of view, all the ereaders can do is, you know, books. Tablets offer chance after chance after chance to sell the user something.

So no, I don’t honestly think dedicated ereader device makers give a shit about the quality of the books that end up on them. In fact, the crappier the ebooks look, the higher the chance that users will turn to tablets. Case in point, I spent time this morning trying to figure out bugs in the eink Kindles. These are bugs that showed up in the latest update. Now for the longest time my Kindle was perfectly stable. Ever since the Fire came out, every update–and there seem to be a lot lately–makes the eink readers work worse and worse. There is no good reason for that. None.

On a side note, I think the Nook is dead. It might be a great device, but B&N has given up on it. The new ‘owners’ (Google? Microsoft?) don’t care because content is king and they aren’t making money off the content going onto a Nook. The only way they can cash in is to lure Nook users into buying tablets. A really nasty sneaky way to do that is to ensure that cross-device apps DO NOT WORK. I think B&N is going to collapse, too. When it goes, that is basically going to leave Amazon and Apple.

I love my eink, but I suspect the bugs are going to get worse while the Fire gets better. Kobo might have a few years while it worms its way into the international market, but in the end the only dedicated readers are going to be shoved into the obsolete closet and there they will stay.

What a state of affairs, eh?

To which Jon replied:

The fact that B&N doesn’t want people using the NC or NT for anything other than what they intend. But these devices are hackable, no matter how hard B&N tries to lock them down. And they do try; every new release of the OS has blocked previous exploits. But those who hack these devices are brilliant. They figure out ways to get around everything, it seems. And that is a good thing! These are OUR devices that we bought and paid for, and we should be able to use them as we see fit.

What shocked me was… well, it was really two things. First, I was surprised at the number of ePub reader applications I found. Most of them are free, which leads into my second shock — just how bad these applications are! I guess one really does get what one pays for. I don’t think I could write an ePub reader (but it might be fun to try), so I can’t imagine why someone would so do and not charge for it, even a paltry $0.99.

It saddens me to think that the Nook may be going away. I really like mine, but I must admit I have thought about saving up for a tablet, though I would really hate to give up my Nook. I really like the new, larger device B&N released, but I haven’t seen anything about its hackability. I looked at it in-store when it first came out and really liked most of the improvements they made to the OS. I asked the rep at the store if the OS would be upgraded for the older devices and he looked at me like I’d just shot a flaming porcupine from my butt. “That’s old tech,” he muttered in defence. More like in ignorance; there was no reason these devices couldn’t run at least 99% of the new OS’s features. I could have argued it with him, but there wouldn’t have been much point to it.

I do think the dedicated ereader makers (at least Amazon and B&N) do care about how ebooks look on their devices. If they look like crap, that reflects badly on their devices. In my opinion. ;)

To which I replied:

You’d think Amazon and B&N would be more concerned, but the evidence says otherwise. Apple is fanatical about quality control, but I suspect it has more to do with protecting the integrity of their devices. Making attractive ebooks is a side effect.

The rest seem happy to let the producers and consumers duke it out. Meaning, it’s up to the producers to play catch up (if they can) and for consumers to keep buying newer, better, fancier devices. It’s weird to me, but I’m not a business person and don’t pretend to be one.

********************************

So that’s it. Something is going on in the land of ereaders and ebooks, but hell if I have a real clue about what it all might mean. My suspicions are two-fold: One) B&N is circling the drain (my bets are on them showing up in bankruptcy court before the year has ended); Two) Dedicated ereader devices are going the way of the 8-track tape player.

I don’t know for a fact what any of this means for me and thee. I probably wouldn’t even know something was up if I didn’t have three models of Kindles so I could watch as one wildly improves (the Kindle Fire) while the others (eink Paperwhite and Keyboard) slowly degrade with every update. Jon’s experience leads me to suspect that nobody cares enough about the Nook to invest in apps and other support for its users.

Make of this what you will.

How The Kindle Works

I talk a lot about broken ebooks, but judging from the search terms that bring visitors to this site, I suspect many people do not know what I mean. Part of the confusion is because I am actually talking about Kindle books (and I should make myself clearer, sorry). Since I don’t own a Nook (and have never played with one), or an iPad or iPhone or Sony reader or magic toaster, I tend to judge ebooks by what shows up on my Kindle.

I suspect there are a lot of people who don’t own Kindles. Maybe one or two who’ve never even seen a Kindle.

Here is Amazon’s dirty little secret regarding its Kindle. As long the ebook is converted to either mobi or prc, the Kindle can read it. (I convert raw documents all the time since I prefer reading on my Kindle over manhandling reams of paper–I just run a Word doc through MobiPocket creator and load it up. I’ve done that with pdf files, too.) I don’t call those ebooks. A person who isn’t aware of how a Kindle actually works, might be unaware their ebook is broken–after all, Amazon let them upload it and reported no problems.

Now, the wise formatter will convert their ebook using KindleGen and check their work on the Kindle Previewer. Amazon just updated it and it has more options, include some font changing, so it’s more reliable now. To truly make sure the ebook works, it should be loaded on a device and run through its paces. People who produce a mobi file using Scrivener or convert a file in Caliber or MobiPocket or run a Word doc through the onsite converter at Amazon might end up with a broken ebook and not know it. (A kind reader might take pity and send you an email to let you know your book is broken, or they might return it for a refund, or–worst of all–they might decide your ebook is too unpleasant to read and not buy any more of your books.)

Pardon my less than stellar photography–here is what the menus look like on Kindles.

Kindle MenusThe font sizes are pretty hard to screw up. There was a Kindle-induced bug that shrunk the font, forcing users with older Kindle models to have to greatly increase the font size in order to read. I think that bug was fixed. But I don’t think I’ve ever run into an ebook where the size can’t be changed. The fonts themselves can be changed.

  • Keyboard: “regular” “condensed” and “sans serif.
  • Paperwhite: Baskerville, Caecilia, Caecilia Condensed, Futura, Helvetica and Palatino
  • Fire: Baskerville, Caecilia, Georgia, Palatino, and Helvetica

A common flaw is a locked font (usually in the ugliest choice). After looking at the html in ebooks that have “locked” fonts, I think what is happening is the producer, using a word processor, has defined a font the Kindle doesn’t recognize. So it displays in the closest match. But, since the font is defined, it can’t be changed.

Line spacing is an option on all models of Kindle. It’s a useful one and it’s also a common “break.” When I format a book in html I don’t mess with line spacing. I define the line height so my text isn’t squished, but that’s different than single-space, space-and-a-half and double space. Word has a really nasty habit of inserting a definition for line-spacing into the document that will override the user menu. Sometimes this is deliberate on the producer’s part, sometimes it is inadvertent because that’s just how Word rolls. In any case, it’s undesirable.

Keyboard line spaceAnother common problem is when the margins don’t work. In the older Keyboard model the user can set how many words there are on a line (fewest, fewer and default) and on the Paperwhite and Fire they can set the margins to narrow, normal or wide. Breaks tend to happen when a producer using a word processor, Scrivener or InDesign justifies the text. Why this affects the margins, I don’t know, but it does.

The Fire allows the user to change the background color. White, sepia or black (sorry, Paul, but black? Oh, my eyes!). It’s a nifty feature, but there is a drawback.

AdjustmentsI apologize for the crappy photo, but if you look very closely at the graphic I circled in red you will see a white box around the graphic. For some odd reason Kindle does not recognize that background is transparent. It’s not a huge issue, but one I hope is soon addressed. Something to keep in mind when using graphic elements in your ebook.

Speaking of graphics… Kindles can be read in landscape mode. The Paperwhite requires an ebook that is specifically coded to be read in landscape mode (such as comic panels or a children’s book–unless, there is some command I am too stupid to figure out and am just missing it) The Keyboard can be changed through the menu and the Fire by turning the device.

LandscapeLandscape mode can have a significant effect on graphics, especially those that are sized to fit the portrait screen. What I do is size the graphics in percentages so that no matter what size the screen or if the book is being read in landscape mode, it will “shrink” or “expand” to fit the text.

(I was reading a novel that had an interesting block graphic in the header. It looked great in portrait mode, but when I flipped it to landscape suddenly it was just a dumb looking box perched atop the text. Yikes!)

Another common problem is page break failure. The best I can tell (and I’m sure there are those smarter than I who will pop in and set me straight) this is a problem when a producer converts an EPUB file into a mobi file through Caliber. Why Caliber destroys the page breaks is anybody’s guess, but it often does.

So what is the poor ebook producer to do? Especially if you do not have a Kindle on which to test your files? (And this isn’t a slam against people who don’t have Kindles–I don’t have an ereader that uses EPUB files, so I’m playing guess and by golly, too. One advantage with EPUB files is that if my file is validated and I haven’t inserted any weird stuff that could override defaults, I’m fairly certain it will work properly. I’d love it if a Nook owner wrote a guest post about its features and common problems. Any takers?)

  • Download the Kindle Previewer and use it. It’s not perfect and you can’t test ALL the device features, but it will give you a far better display than Caliber or even the previewer at Kindle Direct.
  • If you use a word processor, Scrivener or InDesign be very careful with your style sheets. Do not justify the text. Leave the line spacing at single-space. Don’t get fancy with your margin settings. What YOU see is NOT what the end user will get.
  • Experiment with graphic element sizes and use percentages (when possible) rather than fixed em or pixel sizes.
  • Learn html and get away from using not-quite-right for ebooks programs.

So, now you know what I mean when I say “broken” ebooks. EPUB readers, what are the common problems you find?

 

 

Are Ebooks Getting Too Complicated For DIY?

I’m a tad out of sorts this morning. Irked, annoyed, disgruntled… Pick one.

Partly it is because of anxiety. I used Paul Salvette’s guide to create my very first complete ebook (one that doesn’t have to go through a third-party conversion process). Now the author is waiting for it to be published on Amazon. I hate the waiting…

Partly it’s because my son gave me a Kindle Fire for my birthday. Oh, wow, is that thing cool. It’s not something I’d have bought for myself. I’m not a gadget person and it takes me forever to warm up to anything new. But wow, the Fire is very cool. The first thing I did was load the book I just finished onto the Fire to see how it looked. It looks gorgeous. It also looks a lot different than on Larry the Kindle.

I looked at other ebooks in my library. I have books put out by big publishers and indie books, and books that were professionally formatted and books that were DIY. Quality is all over the board. Some of the books that look just fine on Larry look amateurish and not-quite-right on the Fire. It’s because many of the books were produced before the Fire existed. The older formatting platform doesn’t translate so well. The standards are different.

The ebooks are readable. I’m not going to pitch a bitch just because a book I purchased last year won’t let me adjust fonts on the Fire. Nor am I going to ping DIY publishers who’ve formatted a Word file according to Amazon’s guidelines and ended up with an amateurish looking ebook.

I’m irked and annoyed at the devices and the platforms and distributors. Quite frankly, this shit has gotten way too complicated.

It doesn’t help that I read Baldur Bjarnason’s latest post at Futurebook. This part worsened my mood:

However, as I’ve written about before, a large proportion of ebooks published are rubbish. Not in terms of the content (although that’s probably also the case) but in terms of the quality of the file. Ereader platform vendors cannot support the full range of CSS that EPUB2 and EPUB3 require because a substantial number of their catalogue would become unreadable.

Platform vendors are in a position where they couldn’t support standards completely even if they wanted to.

No kidding. For instance, while I was building my most recent project, this is what I had to do. Build the file. Launch the file in my web browser. See how it looks. Figure out why something doesn’t look the way I wanted it to look (All the while knowing that what appears in my browser is only an approximation of what will appear on the ereader). Fix and fiddle, then validate the file to make sure it meets EPUB standards. Check how it looks in Calibre (I don’t have a device that reads EPUB). Again, I know that what I see on my computer screen is not necessarily what a reader will see on a Nook or iPad or whatever. Then, I convert the file into MOBI format and load it onto my Kindle. Do more tweaking. Tweaking and fiddling means having to go through validation again. It means more converting and loading and inspecting. And I haven’t even gone through the Kindle Previewer yet. I want to know how my ebook looks on as many devices as possible. I change font sizes and line spacing and the size of the reading window. It’s time-consuming, it’s frustrating, but the worst part is that even though I’m checking and double-checking with everything I have on hand, it’s still not enough. There is no guarantee that an ebook that renders perfectly on Larry the Kindle (and now the Fire) is going to render properly on other Kindle styles or versions, the Nook, the iPad, the iPhone, an Android, a Sony, a whatever.

As the cat sez:

This is, in a nutshell, a disservice to readers. READERS. Do you hear me Amazon? Barnes & Noble? Kobo? Smashwords? Apple? While you guys indulge in device wars and competing formats while creating compatibility issues, are you thinking about readers at all? You know, the people paying the bills? It’s all well and good to roll out the welcome mat to publishers big and small, traditional and indie, and invite all comers to list their ebooks with you. You get your percentage of sales. When your guidelines and standards are such that it is very, very easy for anybody to make a crappy looking ebook, naturally people are going to follow your guidelines to the letter and end up with crappy looking ebooks.

That’s not right. It’s not fair to readers.

It’s difficult making an ebook that renders properly across all devices. For the self-publisher who has neither the time nor the inclination to learn all ins and outs of formatting to meet different standards, it’s damned near impossible.

That’s unfair to the do-it-yourselfer. It’s unfair to their readers.

What’s the solution? I do not know. I’m not a programmer or a tech-type. I have no idea what goes into creating these devices or how they work. I just want ebooks that respect the material and are a pleasure to read. That is not too much to ask. All this screwing around with fancier devices and increasingly complicated and narrow platforms is making it too damned hard.

Source Files Update

So last week I talked about the importance of writers shifting their mindset from thinking “Print Documents” to thinking  “Electronic Files.” Judging by the responses I got, I’d say I’m not the only one concerned with this subject. One of the problems is that the tools we use–namely word processors–are superb for producing printed documents, but frustrating, maddening and over-powered when creating electronic files.

Currently, I’m in the process of creating a booklet/cheat sheet to help fiction writers who are NOT computer programmers to painlessly use the tools they have–and are comfortable using–to create electronic files suitable for e-queries, e-submissions and ebooks. You’d think this would be simple, but it’s not. For one thing, there’s a language problem. I’m a fiction writer, not a programmer. I think in terms of print. Adjusting my way of looking at the subject is difficult and I backslide worse than a dieter at a chocolate fountain. I’ve been talking to William Ockham and he thinks my efforts are on the right track, but kind of sad, too, because he is a computer programmer and he is frustrated by how out-of-synch users and developers are. He is currently working on a program for ebook producers. Essentially he wants a program similar to what blogging sites use. The user selects a “theme” for how he/she wants the ebook to look, then loads in their text and images and the program takes care of all the formatting. If William succeeds, I’ll be his first customer.

In the meantime, I’d thought I’d show you what we’re up against.

First, here is a document provided by a writer for whom I’m producing an ebook. Those little marks you see in the image indicate spaces and paragraph returns.

The writer has created a document suitable for print. Everything is nicely laid out on the screen and without the Show/Hide feature toggled, one would look at it and think, “Beautiful.” Here’s how it looks if I convert this and load it on my Kindle.

Because the writer used paragraph returns to make a new page (instead of inserted page breaks) this is what the start of chapter two looks like:

Now here is the same document with all the extra paragraph returns and spaces removed, and the special formatting and paragraph returns tagged.

Here is what the same document looks like loaded into a text editor for HTML formatting:

Here is what the same document looks like when I upload a cleaned up copy into Scrivener for ebook formatting:

Here is what it looks like if I format it for an ebook in Word (for uploading to Smashwords):

Here is an end product I’m going for (still some tinkering left to do, but you get the idea):

I had thought (foolishly) that all my problems were solved by getting away from Word (except to use its turbo-charged Find/Replace feature to clean up files) and using Scrivener to produce ebooks. It does a great job and it’s relatively painless. Last week I learned there is a problem with one of the books I produced. The problem occurred because Amazon updated older model Kindles and now some people will find that some ebooks–not all!–are compressed and they will have to manually enlarge the screens. Why? I think it’s something Amazon did, but hey, I’m a fiction writer, not a programmer. I do not know why.

Until I do know why and can figure out how to prevent that from happening, I’m now leery of Scrivener. So, damn it, I’m doing what I told myself I really didn’t want to do, and am learning HTML in order to produce ebooks. Come to find out, there are different languages and coding for HTML, too! Ay yi yi. My eyes aren’t red just from the Colorado wildfires, but from spending hours trying to figure this shit out.

It shouldn’t be this hard. Electronic files should not be filled with tiger traps and landmines waiting to turn our lovely words into gibberish marching all over a user’s screen. Those e-queries and e-submissions we labor over until they are perfect should not turn into funhouse mirror words the moment we hit SEND. Creating an ebook should not require a degree in Computer Science and fluency in programming languages. There shouldn’t be different formats requiring different source file layouts and conversion programs and different requirements from every single e-distributor.

Woulda, coulda, shoulda…

The reality is, this Tower of Cyber-Babel and the Facebook Effect (taking something that works fine, and people enjoy using, and tinkering and “fixing” it until everybody loathes it) are what we have to deal with right now. It is more important than ever for writers to change the way they think about the files they create and how they will be used (e-queries, e-submissions, ebooks, print) and adjust the way they use their favored word processors.

Watch this space. The cheat sheet is slowly becoming a reality and I’ll let you know when it’s ready to go.