Guest Post: Karen Myers–No Need For Calibre

When I ran the recent ebook formatting contest, I noticed some similar issues with many of the entries, all of them the result of Calibre conversion. I have no beef with Calibre, it’s a great online reading platform and I use it myself, but it’s a not-quite-right tool for converting Kindle books. One entrant, Karen Myers (Perkunus Press), contacted me about the problems in her ebook.

KMblog6The flaws weren’t fatal, but Karen was not satisfied with not-quite-right. So I pointed her at Sigil and Paul Salvette’s books, tutorials and website, and now I’ll let her tell you the results:

__________________________

Silly me. I’m an old programmer and I pride myself on trying to get my ebooks “just so”, as if I were writing a piece of code. I want to create worthy offerings to add to humanity’s river of books; at the very least, they should be shiny and well-scrubbed.

So when Jaye offered to judge the formatting of a few books from her blog fans, I hopped right on board, and she was very kind in her review.  But I read with horror things like “squishy line spacing” and links to chapters not working quite as they should, systematically.

I use an EPUB reader and hadn’t seen the book on a Kindle device other than the PC Previewer, so it was useful to see this from the Kindle reader’s perspective, since none of my buyers had complained (yet). Without a Kindle device, I hadn’t realized quite how irritating it was to not properly trigger the “Cover” and “TOC” hard buttons.

Now on the one hand, it wasn’t really broken, but on the other hand, I want perfection in book formatting, and some cosmetic and graphic flourishes. I’m not willing to settle for “good enough,” so Jaye was nice enough to coach me through some of the issues.

(Jaye here: I made a template for Karen based on Paul Salvette’s The Ebook Design and Development Guide)

If you’re content with auto-conversion from EPUB to MOBI or vice versa, or output direct to ebook formats from products like Scrivener, then this is overkill for you and you can stop reading now. But if you want as much control as possible over the results without killing yourself, you might find this approach useful.

Originally I formatted my ebooks in raw HTML using tips from a variety of online recommendations, like Guido Henkel’s. The first book was a real learning curve for me, but after that it wasn’t difficult to just do the same for later books.  Automated search/replace macros took care of things like wrapping lines with <p></p> tags, and so forth.

I took the HTML output, opened it in Calibre, added the cover and converted it to EPUB using Calibre defaults (more or less). I did the same with a separate conversion to MOBI, which required me to maintain a different HTML file because of the way Calibre generates the MOBI TOC. These outputs were what I was uploading to the distributors. The MOBI files produced this way were not ideal, possibly because of AZW vs MOBI choices (in other words, me as a Calibre user, not necessarily Calibre as a tool), and I was left with two files to maintain (EPUB and MOBI) and the Smashwords EPUB as a third file, since my approach wasn’t modular.  So every time I found a typo…

Jaye has taught me a better way… (Sigil)

KM blog1KMblog2I poured my HTML file for a book, broken up into chapters, into a Sigil template.  Each part of my book has a separate file: Beginning Blurb, Title Page, Copyright Page, Also-By-This-Author Page, Small TOC, Chap 1… Chap Last, Guide & Name Index, If-You-Liked-This-Book Page, Excerpt-From-Next-Book Page, Author Bio Page, Long TOC.

KMblog3I have 3 outputs: MOBI, EPUB, Smashwords EPUB. The difference between Mobi and the two EPUBs is that the “stylesheet.css” file is a little different between MOBI and EPUB, and the Cover page is treated differently (EPUBs require an extra step). The difference between my EPUB and my Smashwords EPUB version is that the Copyright Page has different content.

Now, thinking in the long term, I expect that the differences between MOBI output and EPUB output are likely to be persistent, and other devices may come along and generate different optimum stylesheet requirements. So I’m fine with having two different (but very similar) stylesheets which I maintain externally and copy in as needed into the stylesheet.css shell.

Likewise, the fact that Smashwords requires its own ISBN means only that I maintain two different external Copyright Page HTML documents and copy the contents of whichever one I want into the shell in Sigil.

Both of these make use of a simple modular structure.

So, what happens when I finish a new book and want to format it?

PREP

  1. I copy the Sigil file (MOBI version) from the previous book and rename it.
  2. I create Copyright.HTML files for both the normal and Smashwords copyright pages by copying the ones for the last book, renaming them, and updating the content.
  3. I create a new Title page (it’s a graphic) and a new Cover.

KMblog4MAIN CONTENT (MOBI)

  1. I work on the MOBI version first (it’s the master). I copy the text in, chapter by chapter, the front blurb, and the back excerpt.  I run saved searches to wrap the lines with <p></p> tags and to convert special characters to named entities.
  2. I update the Also By and If You Like This book pages by hand.
  3. I run a saved search to update the Title field on all the HTML pages to the new work, and update the equivalent fields in the TOC.NCX and CONTENT.OPF files.
  4. If the book is a little longer or shorter (number of chapters) than the last one, I update the TOC.NCX and CONTENT.OPF files and the HTMLTOC file.
  5. I update the metadata in the TOC.NCX and CONTENT.OPF files. This allows me to do some things that either Calibre doesn’t, or I don’t know how to find, such as set a UUID (Unique User ID) for my short stories that don’t have ISBNs, embed book descriptions, add keywords, etc.  There’s a great tool for this that Jaye told me about:

bb ebooks meta-pad generator (courtesy of Paul Salvette, BB Ebooks Thailand)

KMblog5Run the file through Kindle Previewer (which runs Kindlegen) and check the results.

How much time does this take?  I just updated my entire backlist (3 novels, 5 short stories, 1 story collection) to Sigil–it takes me about an hour for a novel, and 20 minutes for a short story.  Making the short story collection from the already-formatted short story files was truly trivial.

EPUB VARIANT

  1. Substitute the content of the EPUB stylesheet into the stylesheet.css.
  2. Run the Sigil tool to insert the cover.  (Kindlegen does that a different way for MOBI).
  3. For the Smashwords variant, substitute the contents of the Smashwords copyright.html for the default one.

That’s it.  I import the files into Calibre for one last look to make sure they seem healthy, and do a quick scroll through on my EPUB reader. If I find typos, I fix them in the MOBI version (and the Scrivener original) and redo steps (10)-(12).

Why not use Calibre?  I am confused by the various options and clearly, for MOBI conversion, I wasn’t doing it quite right. Also, my original HTML file was one big file with a stylesheet and all chapters together, making modular changes clumsy. Calibre created its own version of the styles it found, and they weren’t always what I expected. It’s a big black box to me, and there were some issues with the results, which may be my fault, not Calibre’s.

Why not use Scrivener?  Like Calibre, you are at the mercy of whatever Scrivener decides to do to instantiate the different conversions. Since the Scrivener text isn’t in HTML, there are all the issues of named entity conversions to deal with, and you have little control over the default styles. The results may be clean, but you can’t do anything special, such as use graphic chapter heads, scene dividers, and so forth, at least not in the Windows version.  Perhaps there’s a way…

Other tools, like Scrivener, will take your word processor input and generate EPUB and MOBI output, but the black box in between what you write and what they produce leaves you at the mercy of the limitations of others, and so your output will remain at best functional vanilla. That’s not a bad thing, but we can do better.

It’s really not that hard to go through the learning curve once.  After that, each new book becomes quite easy.  Your book designers or people like Jaye can help you get started by setting up the first one and explaining how it works.

_____________________________

Jaye again. Thank you, Karen. Now I’d like to add a word about Sigil.

Sigil creates EPUB files. Kindle ebooks are MOBI files converted from EPUB files. So, you can take an EPUB file created in Sigil and convert it with KindleGen into a MOBI file that will work on Kindles.

There’s a gotcha–One size does NOT fit all. EPUB and MOBI handle styling differently. They handle covers differently. If you know what the differences are, you can use Sigil to create ebooks that work exactly how they are supposed to, on EPUB and Mobi. You can control how the devices handle your work, as opposed to being at the mercy of whatever the conversion program decides is best.

I have a special offer for readers of this blog. If you are really serious about creating ebooks that work properly across devices, and you are ready to climb the next step on the learning curve, go buy Paul Salvette’s The Ebook Design and Development Guide. It might freak you out the way it freaked me out when I first read it (I have zero programming experience). If it does, but you’re still motivated, send me an email (jayewmanus at gmail dot com) with a screenshot of your receipt for Paul’s book and I’ll set you up with a template similar to the one I made for Karen.

 

 

TWO Files For Smashwords?!? Not So Fast With The WTF, Folks

I’ve been one of the noisy gripers bitchin’ about the Smashwords “Meatgrinder.” My complaint was not what Mark Coker of SW was doing, but that MS Word makes lousy ebooks. Now, Coker has made it possible for ebook producers to submit validated EPUB files for distribution wherever fine EPUB-platform ebooks are sold.

This is terrific news.

Now I’m seeing complaints all over the ‘net that in order for an ebook to be fully distributed in the SW catalog one must also submit a Word file along with the EPUB file. A lot of WTF going on and people acting as if they’ve been somehow buffaloed.

Back off a minute and put down your pitchforks and torches. In order for SW to do what it’s been doing, it’s had to take a one-size-fits-all approach (could not have afforded it any other way). Using Word as the source file for conversion made sense for two reasons:

  • One) SW is mostly a self-publishing platform for WRITERS who use WORD PROCESSORS to create DOCUMENTS;
  • Two) Ebook files are based on html coding (they are essentially little websites) and most word processors are based on html which can be converted so they can be read on various and sundry devices.

The problems were not so much in the conversion. The problems came from the ereader devices. Every one of them is different. Some use older technology, some use the newest technology. Many have user interfaces, allowing readers to customize (to an extent) the way they read an ebook. (Ever wonder why mobi files are so big compared to an EPUB file? It’s because they are actually several different formats–eink, tablet, keyboard, touch screen–all of which display differently and give the reader different options on the various Kindle devices.)

Smashwords also offers readers different options, such as PDF and (essentially) text files for reading on the computer. They offer formats like LRF and PDB for people with older, almost obsolete devices.

A mobi file can be converted from EPUB, but it requires some adjustments to the css, the cover image and navigation coding. You can do things on a Nook you can’t do on a Kindle (for instance), and vice versa. Much different platforms. I can convert an EPUB to a mobi file and read it on my Kindle, but in order to make it work properly on all Kindle devices, in order to make it convert through Kindlegen without errors, I need a different type of EPUB file.

Then you get into the platforms that aren’t based on EPUB at all. Can I convert an EPUB file into a pdf file? Well, sure, but it’s ridiculously convoluted and requires more clean-up than conversion. The reason is in the name: “Portable Document Format.” Word files convert easily into pdf files because both of them are document files.

The beauty of what Smashwords has done is that if you have a validated EPUB file (and that means error free according to IDPF–International Digital Publishing Forum–standards) it is going to work on the various devices using the EPUB platform–namely Nook, Kobo and Apple products. It will work the way users (our customers) want them to work and the way the device makers intend them to work.

What it boils down to is quality control. I can control the quality of EPUB files in ways that are not possible with a Word file. It’s not about the bells and whistles, it’s about the formatting and making sure my ebooks are stable and functional across devices.

If you understand how ebooks work and how other file formats work, then you know it is not feasible for SW to convert EPUB files into other formats such as mobi or pdf or rtf. That’s my job. These are my ebooks and my readers/customers, and it’s up to me to figure out the best way to make the ebooks I create compatible with their devices.

There is no “one-size-fits-all” when it comes to ebooks and ereading devices. SW made a valiant effort when it tried to force Word into that role, but it was doomed from the get-go because Word is not the right tool.

EPUB is only one format out of many, and it is not Smashword’s or Mark Coker’s fault that the retailers and device makers cannot get their shit together and settle on a standard.

You do not have to submit two files to SW if you don’t want to. You can go EPUB only–which shuts out those who don’t have a device based on the EPUB platform. You can submit a Word file only–take your chances that your ebook is going to glitch, or settle for an ebook so generic it might as well be a text file.

Something else, too. Smashwords is a distributor. It reaches markets that indies cannot always reach on their own. I suspect the number one reason many of those avenues are closed to direct distribution from indies is because those outfits don’t want to deal with buggy, broken, half-assed ebook files created in word processors. SW could have insisted that those who wished to use their distribution service must provide files in compliance with the different platforms. That would have set back the ebook revolution several years. Instead SW came up with concept that mostly worked. So to those who are bitching that they now have to provide TWO files to SW, take a deep breath, step back and consider the alternative–the market could demand that you create up to ten different formats in order to reach all your potential readers. That, my friends, would be real cause for cries of WTF.