Guest Post: Karen Myers–No Need For Calibre

When I ran the recent ebook formatting contest, I noticed some similar issues with many of the entries, all of them the result of Calibre conversion. I have no beef with Calibre, it’s a great online reading platform and I use it myself, but it’s a not-quite-right tool for converting Kindle books. One entrant, Karen Myers (Perkunus Press), contacted me about the problems in her ebook.

KMblog6The flaws weren’t fatal, but Karen was not satisfied with not-quite-right. So I pointed her at Sigil and Paul Salvette’s books, tutorials and website, and now I’ll let her tell you the results:

__________________________

Silly me. I’m an old programmer and I pride myself on trying to get my ebooks “just so”, as if I were writing a piece of code. I want to create worthy offerings to add to humanity’s river of books; at the very least, they should be shiny and well-scrubbed.

So when Jaye offered to judge the formatting of a few books from her blog fans, I hopped right on board, and she was very kind in her review.  But I read with horror things like “squishy line spacing” and links to chapters not working quite as they should, systematically.

I use an EPUB reader and hadn’t seen the book on a Kindle device other than the PC Previewer, so it was useful to see this from the Kindle reader’s perspective, since none of my buyers had complained (yet). Without a Kindle device, I hadn’t realized quite how irritating it was to not properly trigger the “Cover” and “TOC” hard buttons.

Now on the one hand, it wasn’t really broken, but on the other hand, I want perfection in book formatting, and some cosmetic and graphic flourishes. I’m not willing to settle for “good enough,” so Jaye was nice enough to coach me through some of the issues.

(Jaye here: I made a template for Karen based on Paul Salvette’s The Ebook Design and Development Guide)

If you’re content with auto-conversion from EPUB to MOBI or vice versa, or output direct to ebook formats from products like Scrivener, then this is overkill for you and you can stop reading now. But if you want as much control as possible over the results without killing yourself, you might find this approach useful.

Originally I formatted my ebooks in raw HTML using tips from a variety of online recommendations, like Guido Henkel’s. The first book was a real learning curve for me, but after that it wasn’t difficult to just do the same for later books.  Automated search/replace macros took care of things like wrapping lines with <p></p> tags, and so forth.

I took the HTML output, opened it in Calibre, added the cover and converted it to EPUB using Calibre defaults (more or less). I did the same with a separate conversion to MOBI, which required me to maintain a different HTML file because of the way Calibre generates the MOBI TOC. These outputs were what I was uploading to the distributors. The MOBI files produced this way were not ideal, possibly because of AZW vs MOBI choices (in other words, me as a Calibre user, not necessarily Calibre as a tool), and I was left with two files to maintain (EPUB and MOBI) and the Smashwords EPUB as a third file, since my approach wasn’t modular.  So every time I found a typo…

Jaye has taught me a better way… (Sigil)

KM blog1KMblog2I poured my HTML file for a book, broken up into chapters, into a Sigil template.  Each part of my book has a separate file: Beginning Blurb, Title Page, Copyright Page, Also-By-This-Author Page, Small TOC, Chap 1… Chap Last, Guide & Name Index, If-You-Liked-This-Book Page, Excerpt-From-Next-Book Page, Author Bio Page, Long TOC.

KMblog3I have 3 outputs: MOBI, EPUB, Smashwords EPUB. The difference between Mobi and the two EPUBs is that the “stylesheet.css” file is a little different between MOBI and EPUB, and the Cover page is treated differently (EPUBs require an extra step). The difference between my EPUB and my Smashwords EPUB version is that the Copyright Page has different content.

Now, thinking in the long term, I expect that the differences between MOBI output and EPUB output are likely to be persistent, and other devices may come along and generate different optimum stylesheet requirements. So I’m fine with having two different (but very similar) stylesheets which I maintain externally and copy in as needed into the stylesheet.css shell.

Likewise, the fact that Smashwords requires its own ISBN means only that I maintain two different external Copyright Page HTML documents and copy the contents of whichever one I want into the shell in Sigil.

Both of these make use of a simple modular structure.

So, what happens when I finish a new book and want to format it?

PREP

  1. I copy the Sigil file (MOBI version) from the previous book and rename it.
  2. I create Copyright.HTML files for both the normal and Smashwords copyright pages by copying the ones for the last book, renaming them, and updating the content.
  3. I create a new Title page (it’s a graphic) and a new Cover.

KMblog4MAIN CONTENT (MOBI)

  1. I work on the MOBI version first (it’s the master). I copy the text in, chapter by chapter, the front blurb, and the back excerpt.  I run saved searches to wrap the lines with <p></p> tags and to convert special characters to named entities.
  2. I update the Also By and If You Like This book pages by hand.
  3. I run a saved search to update the Title field on all the HTML pages to the new work, and update the equivalent fields in the TOC.NCX and CONTENT.OPF files.
  4. If the book is a little longer or shorter (number of chapters) than the last one, I update the TOC.NCX and CONTENT.OPF files and the HTMLTOC file.
  5. I update the metadata in the TOC.NCX and CONTENT.OPF files. This allows me to do some things that either Calibre doesn’t, or I don’t know how to find, such as set a UUID (Unique User ID) for my short stories that don’t have ISBNs, embed book descriptions, add keywords, etc.  There’s a great tool for this that Jaye told me about:

bb ebooks meta-pad generator (courtesy of Paul Salvette, BB Ebooks Thailand)

KMblog5Run the file through Kindle Previewer (which runs Kindlegen) and check the results.

How much time does this take?  I just updated my entire backlist (3 novels, 5 short stories, 1 story collection) to Sigil–it takes me about an hour for a novel, and 20 minutes for a short story.  Making the short story collection from the already-formatted short story files was truly trivial.

EPUB VARIANT

  1. Substitute the content of the EPUB stylesheet into the stylesheet.css.
  2. Run the Sigil tool to insert the cover.  (Kindlegen does that a different way for MOBI).
  3. For the Smashwords variant, substitute the contents of the Smashwords copyright.html for the default one.

That’s it.  I import the files into Calibre for one last look to make sure they seem healthy, and do a quick scroll through on my EPUB reader. If I find typos, I fix them in the MOBI version (and the Scrivener original) and redo steps (10)-(12).

Why not use Calibre?  I am confused by the various options and clearly, for MOBI conversion, I wasn’t doing it quite right. Also, my original HTML file was one big file with a stylesheet and all chapters together, making modular changes clumsy. Calibre created its own version of the styles it found, and they weren’t always what I expected. It’s a big black box to me, and there were some issues with the results, which may be my fault, not Calibre’s.

Why not use Scrivener?  Like Calibre, you are at the mercy of whatever Scrivener decides to do to instantiate the different conversions. Since the Scrivener text isn’t in HTML, there are all the issues of named entity conversions to deal with, and you have little control over the default styles. The results may be clean, but you can’t do anything special, such as use graphic chapter heads, scene dividers, and so forth, at least not in the Windows version.  Perhaps there’s a way…

Other tools, like Scrivener, will take your word processor input and generate EPUB and MOBI output, but the black box in between what you write and what they produce leaves you at the mercy of the limitations of others, and so your output will remain at best functional vanilla. That’s not a bad thing, but we can do better.

It’s really not that hard to go through the learning curve once.  After that, each new book becomes quite easy.  Your book designers or people like Jaye can help you get started by setting up the first one and explaining how it works.

_____________________________

Jaye again. Thank you, Karen. Now I’d like to add a word about Sigil.

Sigil creates EPUB files. Kindle ebooks are MOBI files converted from EPUB files. So, you can take an EPUB file created in Sigil and convert it with KindleGen into a MOBI file that will work on Kindles.

There’s a gotcha–One size does NOT fit all. EPUB and MOBI handle styling differently. They handle covers differently. If you know what the differences are, you can use Sigil to create ebooks that work exactly how they are supposed to, on EPUB and Mobi. You can control how the devices handle your work, as opposed to being at the mercy of whatever the conversion program decides is best.

I have a special offer for readers of this blog. If you are really serious about creating ebooks that work properly across devices, and you are ready to climb the next step on the learning curve, go buy Paul Salvette’s The Ebook Design and Development Guide. It might freak you out the way it freaked me out when I first read it (I have zero programming experience). If it does, but you’re still motivated, send me an email (jayewmanus at gmail dot com) with a screenshot of your receipt for Paul’s book and I’ll set you up with a template similar to the one I made for Karen.

 

 

Advertisements

16 thoughts on “Guest Post: Karen Myers–No Need For Calibre

  1. Great article, Karen. For you programming nerds following along, here’s the code (borrowed from Stack Overflow) to make UUIDs in Javascript:

    function genUuid() {
    return ‘xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx’.replace(/[xy]/g, function(c) {
    var r = Math.random()*16|0, v = c == ‘x’ ? r : (r&0x3|0x8);
    return v.toString(16);
    });
    }

    And in PHP:
    function gen_uuid() {
    return sprintf( ‘%04x%04x-%04x-%04x-%04x-%04x%04x%04x’,
    // 32 bits for “time_low”
    mt_rand( 0, 0xffff ), mt_rand( 0, 0xffff ),

    // 16 bits for “time_mid”
    mt_rand( 0, 0xffff ),

    // 16 bits for “time_hi_and_version”,
    // four most significant bits holds version number 4
    mt_rand( 0, 0x0fff ) | 0x4000,

    // 16 bits, 8 bits for “clk_seq_hi_res”,
    // 8 bits for “clk_seq_low”,
    // two most significant bits holds zero and one for variant DCE1.1
    mt_rand( 0, 0x3fff ) | 0x8000,

    // 48 bits for “node”
    mt_rand( 0, 0xffff ), mt_rand( 0, 0xffff ), mt_rand( 0, 0xffff )
    );
    }

    A lot cheaper than an ISBN!

  2. Pingback: Perkunas Press

  3. You know, I thought I was proficient at programming until I started getting into ebook coversion. *sigh* I actually bought Salvette’s book a few months ago on your recommendation, and found it terribly intimidating. 😦 But I’ll go back to it and try again! I really do not feel the need for many bells and whistles but I’m very, very intent on my future books looking professional and of high quality. So thanks for this, and thanks Karen for sharing your experiences!

    • It’s worthwhile to try again, Kim. One thing I have noticed is this, if you put your focus on making the ebook work properly, you’ve won 95% of the battle to win readers’ appreciation. I know in my own reading and from listening to others that readers are getting increasingly fed up with ebooks that don’t let them use all the features on their devices. Once you figure out how to make a working ebook, the bells and whistles are easy (and fun).

  4. Great stuff here! Thanks to Karen and Jaye.

    One thing: I may be (may?) misremembering this, but I believe that Sigil automagically assigns a UUID each time it is opened; opening a saved ePub will, of course, use the saved UUID.

    And, if not, Paul’s code is great, too!

    • I believe you are correct, Jon. I only use Sigil for the validation and tweaking the styles, so my ebooks are set up by the time I open them up there. I could check myself, but am running behind, as usual, and don’t have time to crack up an ebook and peek: does Sigil insert meta-data?

      • Hi Jaye:

        Yes, the metadata are completely exposed through Sigil. I believe the Metadata Editor is found under the Tools menu item. Otherwise, one can open the “content.opf” file and hack away.

        Also, to Karen, Sigil has the built-in ability to encapsulate every paragraph pasted into the WYSIWYG editing screen with the “p” tags. This might save you a bit of time when preparing your next eBook.

  5. I’m working my way through this process, too. Programming is part of my day job and I still find all that undefined code intimidating. (Well, undefined only because I don’t understand all the words used to define it.) I’m operating on blind faith that it will all come out okay, and that some day I might actually understand what it all does.

  6. For you authors, just remember, you should take calibre into mind when formatting. I know I’d never have bought a kindle if it wasn’t for calibre since it allows me not only to read books, but comics, fan fiction, manga, and anything else I could desire to read. Honestly I don’t know of a single book (other than those I get from Amazon directly) that isn’t usually just in pdf. Which doesn’t require any conversion, unless there’s a need to increase or decrease font size. And honestly, even as a big reader, out of the 1600 files in my calibre list, only about 300 are actual books. So take heed, novels may just be a tiny portion of what a reader is into, and that may force them to not care about what calibre may do to the formatting of a book (you should see what can happen from epub to mobi and then to pdf, it can get ugly). Plus don’t forget, not everyone uses the same type of device, or even the same kindle, heck I have 4 of them now, DX, paperwhite, k3, and a fire.

    Calibre lets people read things other than novels on their devices, and if calibre mangles your book during conversion, most readers may not care. It’s not just mobi, its azw3, lit, fb2, lrf, or pdf. Despite you making the perfect epub and mobi, it may not be a perfect pdf or whatever, but as long as it can be read, most readers really wont mind. Might make you scream in frustration, but most readers will do whatever they have to in order to read a book. And if it means doing crazy stuff to read a book from amazon on their nook or their 10 year old whatsit, they’ll do it.

    Readers gonna read.

  7. Pingback: Ebook formatting with Sigil (for the control freak) | Hollow Lands

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s