Category: 6. Book Polishing

EPUB image formatting

In an earlier post, I popped Aladore’s illustrations into the EPUB using Sigil with out any clever formatting.  If we take a peak at the HTML contained in the EPUB, each image looks like this:

<p><img alt="aladore_01" src="../Images/aladore_01.jpg" /></p>

The HTML is just like a web page, the img tag tells the ereader to display an image and gives the relative location for the file.  In the example above, the ereader looks for an image called “aladore_01.jpg” in the “Images” directory of the EPUB package.  However, you will notice that there is nothing in this set of tags to tell the ereader how how to handle the image.  For example, how big should the image be?  Does the text wrap?  Does it get its own page? etc.

Now, for many ereader devices and applications, illustrations inserted this way will display just fine–the application automatically resizes all images to fit on the screen.  But, unless you actually put something in the tag you can not be sure… Since the average ereader screen is only around 600 x 800 pixels, many of the illustrations I added will likely run off the screen, looking something like this:

badly sized image

Instead, we want the image to flexibility resize to fit any screen.

This is slightly more complicated than it sounds, since it is not just about using the correct HTML tags–you also have to ensure that ereader devices will play nice with them.  Getting it right across platforms is complicated.  For example, EPUB formatting expert Liz Castro explains intricacies of the img tag in relation to iBooks in a post from 2011.

I read a lot of information about how to deal with this issue, but I had trouble finding anyone who clearly and definitely communicated a solution.  So I decided to look inside professionally published EPUBs to see what they were doing–reverse engineering!

This task could be done using Sigil, but there is another tool uniquely suited to browsing through multiple EPUBs and checking out their inner workings–allow me to introduce Calibre:

Calibre, http://calibre-ebook.com

Calibre is a Free software ebook management system–basically it does everything you could possibly want to do with an ebook…  It manages your library and metadata, converts formats, syncs with devices, and has a built in reader and online book search.  But wait there’s more!  Calibre also has a built in feed reader that can download news and RSS feeds and convert the content into an ebook (and send it to your device automatically).  Additional features and modifications can be added via an extensive plugins system. And now, it even has a built in, full featured ebook editor to rival Sigil!

I have used Calibre for many years to manage my ebook collection.  It sounds like a ridiculous number of features… so practically speaking what does Calibre do?  Basically, I would say its like an ebook version of the mp3 “media center” applications we are all familiar with (i.e. like iTunes or Windows Media Center, but open and awesome). When I download a new ebook, I add it to Calibre.  Calibre copies the ebook and puts its into a directory called Calibre Library (in a sub directory for the author and title).  When I plug in my ereader, Calibre recognizes it.  It shows me what books are currently on the reader.  I can copy, delete, or add books.  If I have a ebook that is not in the correct format, Calibre will specially convert it for the device.

The main developer Kovid Goyal started Calibre back in 2006 after he purchased a SONY PRS-500 e-ink reader.  Maybe this is why I like it so much–I am also an SONY reader-reader!  The “libre” in the name stands for Free software, so I have always called it “ca-libre”–but I found out from the history page that this is wrong!  Kovid wants it pronounced “cali-ber”.  I kind of like the wrong way better…

Anyway, back to the purpose of this post: I browsed through my Calibre library and took a look at some well done commercial EPUBs.  Find a nicely formatted book, then click Edit Book to open the built in editor:

calibre editor

Calibre Editing Aladore.

The features and interface are similar to Sigil.  By default, the code view is in the center with a WYSIWYG preview in the right panel.

Surveying my commercial ebooks, it seems there was two main approaches:

1) use small images that will fit on any screen (less than 300 px) and use css styling to center them.

2) nest the img tag inside a div tag and use styles to set the image to a size relative to the div and the div size relative to the screen.

Option one is basically what we already have in the draft Aladore EPUB–except the images I added are slightly too large to fit on many ereader screens.  The advantage of the smaller images is simplicity, smaller file size, and quicker load time on weak devices.  The disadvantage is that it will not look as good on more advanced ereader devices (as screens get higher and higher resolution) or tablets with larger screen dimensions.

Option two is a bit more complex, but is ultimately more flexible.  It removes the risk that images will be partly cut off on smaller screens and ensures that all devices will handle the images in the same way.  It allows us to add larger images providing better quality on devices that support it.  Here is an example of option two looks like in HTML tags:

<div style="text-indent:0;text-align:center;margin-right:auto;margin-left:auto;width:99%;page-break-before:auto;page-break-inside:avoid;page-break-after:auto;"><div style="margin-left:0;margin-right:0;text-align:center;text-indent:0;width:100%;"><p style="display:inline-block;text-indent:0;width:100%;"><img alt="aladore 1" src="../Images/aladore_01.jpg" style="width:99%;"/></p></div></div>

Looks crazy, but it works… We will revisit this soon!

Book Polishing

Attentive readers may have noticed we broke into a new section of the project with the previous post… oh the excitement!

“6. Book Polishing” is about taking the draft EPUB from a package of raw HTML text and images to something that is actually a Book.  This means working out some ideas about how to properly present the text–formatting, layout, front matter, etc etc.  Hard stuff that takes some practical and artistic thought.  With ebooks we have to make all the typographical and design choices, but also ensure that diverse devices will be able to properly represent the final product as intended.

Oh yeah, and this step at Digital Aladore has nothing to do with the Book Polishing project!

Still Working on Aladore!

Okay, so despite the lack of Aladore related posts for awhile, Digital Aladore is still working on the final EPUB!

I just decided to take some more time to think about the final formatting and polish.  However, lets say you are really, really dieing to read my new text, you could go read it where I posted it on Juxta… just kidding, here is a downloadable plain text version, saved in ODT (WordPress doesn’t let me post txt, sorry for the silly work around):

Aladore_Henry_Newbolt_txt

Or if you prefer, here is a very basic HTML version, again saved in ODT:

Aladore_Henry_Newbolt_HTML

I offer these unformatted versions in case you want to do something with the plain text, like use it for textual analysis or something else fun!  For example you could:

Do some text analysis with TAPoR ware, http://taporware.ualberta.ca/~taporware/textTools

Explore the text with visualizations at Voyant, http://voyant-tools.org

Or try out some serious visual analytics using Jigsaw, http://www.cc.gatech.edu/gvu/ii/jigsaw

(They are all free!)

Let me know if you come up with something interesting!

 

Aladore Regex

In earlier posts, I mentioned using advanced features of find & replace to do some automated editing of the HTML tags in the text.  Let me give you another example to show how useful some clever find & replace can be!

Before uploading the draft Aladore EPUB, I wanted to quickly edit the format of the chapter titles.  At this point the chapter headings looked like this:

<p>CHAPTER IV.</p>

<p>HOW YWAIN CAME TO AN HERMITAGE IN A WOOD.</p>

They look exactly the same as normal body paragraphs.  Of course, in the print version this is not the case.

Aladore 1914, page 20.

Aladore 1914, page 20.

To better match the look of the print book, I decided the headings should be centered and tagged h2.  This will more clearly set them off from the body text, something like this:

<h2 style="text-align: center;">CHAPTER IV.</h2>

<h2 style="text-align: center;">HOW YWAIN CAME TO AN HERMITAGE IN A WOOD.</h2>

However, I also wanted to quickly generate a table of contents.  If I tagged the heading as shown above, the automatically generated TOC would have two separate entries for each chapter.  So a quick and dirty alternative is to put a <br /> between the two parts of the chapter title, making them a single h2 unit that will appear correctly in the TOC.  This solution looks like:

<h2 style="text-align: center;">CHAPTER IV.<br />HOW YWAIN CAME TO AN HERMITAGE IN A WOOD.</h2>

So, how can we automatically find the 58 chapter headings and get them tagged correctly without doing it by hand?  Luckily, Sigil supports Find & Replace with Regular Expressions (i.e. Regex, make sure you have regex chosen for the Mode in the find window). With a few handy expressions and some logical thinking, we can sort this out in no time.  If you want to learn about and practice Regex, check out RegExr.

The main things we need to work with for this application are the lookahead and lookbehind expressions.  In this case it is very easy to test the accuracy of the expression–click Count All and it should be exactly 58 items, otherwise you are not catching only/all the chapter headings.

Sigil find & replace.

Sigil find & replace.

First, I need to replace the <p> in front of the title “CHAPTER…” with <h2 style=”text-align: center;”>.  To do this, use a lookahead, (?=ABC).  This means we are using “CHAPTER” in our search, but will not select it for the Replace function.  The Find looks like this:

<p>(?=CHAPTER)

It will find and select only the <p> that appear before “CHAPTER”, but it does not select CHAPTER.  Awesome!

Next, I want to find the p tags between the two sections of the chapter heading and replace them with <br />.  Since the roman numerals that follow CHAPTER make it a variable string length, for technical reasons we can not use a lookbehind.  Instead we need to figure out a regex that will find only the chapter subtitle (for example, HOW YWAIN CAME TO AN HERMITAGE IN A WOOD.) to use in another lookahead.  Every chapter subtitle is in all uppercase and at least 20 characters–AND there is no other string in the book that has those qualities.  We can use these facts to create our find string: [A-Z,\s,’]{20,}.  This expression means find any string that includes ONLY the uppercase letters A through Z, \s white spaces, or ‘ (needs to be included since some titles have possessives) AND that is 20 characters or more in length.  I didn’t do any serious analysis to decide on the number 20–I just looked a few of the subtitles, counted the number of characters, and tested a few numbers by entering the expression and clicking Count All.  If I got 58 results, I knew I was on the right track.  The expression will find only our chapter subtitles, so we can use it in a lookahead to select the two p tags between the chapter number and the subtitle.  The find looks like this:

</p>

<p>(?=[A-Z,\s,']{20,})

It will select only the two p tags, which I replace with a break.

Finally, we need to replace the </p> after the chapter subtitle with a <h2>.  This time we need to use a lookbehind (?<=ABC), because there is no consistent string after the subtitle that we can use in searching.  In a lookbehind, we can not use a string of open length as I did above.  However, the same qualities of the subtitle will give us enough information to create a distinctive search that will exclude everything NOT a subtitle.  In this case, a string of three characters, including only uppercase letters and a period [A-Z,.]{3} in front of a </p> will give us the 58 results we want.  This is because no other paragraph in the book ends with a word in uppercase letters.  The lookbehind Find expression looks like this:

(?<=[A-Z,.]{3})</p>

It selects only the </p> tag, which I replace with </h2> to close off the heading.  Now, we have all the chapter titles tagged correctly and looking… well, sort of beautiful.  When we get into more polishing, we will use similar expressions to add the style tags needed for CSS.

Isn’t it amazing that we have gone from textual transmission to Regex?  Ha, ha, I love Digital Aladore!

Polishing Plans

Digital Aladore has slowed down a bit… its been a bit busy with other things!

I am thinking more about how to best polish the draft EPUB.  Here are the basic ideas:

  • Better represent the sections of Verse in the text.
  • Better format the text body and chapter headings.
  • Create HTML table of contents at the beginning.
  • Simplify the image formatting.
  • Better cover image.

I would also like to explore the difference between EPUB 2 and 3 to see if the ebook should be updated to the latest standards.

Finally, I would like to test the EPUB on a variety of readers.  So far I have been testing it on my SONY PRS-T1, it would be good to get some impressions on other devices.

Don’t expect anything too soon… sorry!

If you are bored, here is Aladore loaded up in Voyant Tools, so you can explore the word trends… exciting!

 

Ebook UX

The main objective of this section of the Digital Aladore project is to create a polished EPUB2 edition optimized for ereader devices.

Anytime we are designing something, from websites to houses, we should be thinking about the end users. How will the features of the object support an enjoyable and efficient user experience? Even in a very informal design context (such as here in the one man Digital Aladore UX Design Center) our products are molded by how we imagine the user–in this case, the potential readers.

My user model is based on my own experience (as an avid reader of early fantasy novels and user of an ereading device), informal feedback from friends, and MobileRead forum content. I imagine users of the Aladore ebook will be focused on reading the novel in a linear fashion, yet are not interested in strict pagination (like a PDF). Instead they would value clear, readable text reflowing and customizable to suit their particular device. However, they also do not want to be distracted by the technology or innovative features of the ebook. They have expectations of a document genre based on experience reading print novels. They expect the design of the book to reflect the character of the content, or at least not clash with it. For example, take a look at Middlemarch using the Terminal CSS style from EPUB Zen Garden:

Terminal CSS, EPUB Garden.

Terminal CSS, EPUB Garden.

EPUB Zen Garden was inspired by CSS Zen Garden, but unfortunately the site is no longer live (the designs are reproduced in the epub-zen project on GitHub).  A user who selects the Terminal style is likely looking for amusing novelty, not efficient reading or the traditional experience of the novel.  The green screen style is unexpected, distracting, and changes the atmosphere. Furthermore, this style would not function on an greyscale e-ink reader.

Designing for ebooks is complicated by the fact that users will have a diverse mix of devices for reading. Since the function of dedicated ereaders is fairly simple, users update their device less frequently than phones or tablets. I imagine users, such as myself, with e-ink readers that have lower end specs from five or more years ago. Since a focus of Digital Aladore is openness and sharing, the ebook should not exclude these users. This suggests the need for flexible design that supports a wide range of screen sizes from phone to tablet, screen types from e-ink to retina display, and hardware specs. Since devices with larger screens and more powerful processing can use PDF versions of Aladore, this project focuses on optimizing the ebook for e-ink readers. I imagine users having devices such as Sony Reader, Kobo Touch, or Nook Simple Touch in addition to newer e-ink models.

Unfortunately, ereading devices and applications have very inconsistent support for the EPUB specification, which means it is hard to create designs that will behave consistently. Some devices ignore all CSS, only render a small subsection of the styles, or override them by default. Most devices also allow users to change style settings such as font and font size. For example, the native Reader app on Barnes & Noble Nook overrides any included styles by default, setting its own font, font size, line spacing, text-indent, and margins. The user must open the options and toggle “Publisher Defaults” on to view the ebook with its built in styles.  This means even if you spend time creating the perfect XHTML and CSS markup, the reader may never see your design!

While there are significant challenges to presenting a consistent representation of the document to users, good design of the ebook will still support user choice, accessibility, and usability.

Having a solid markup will create a sustainable and flexible product that can “play nicely” with the world of web standards. In a presentation titled “A Cautionary Tale About Poor Ebook Markup” (Ebookcraft 2014), Liza Daly gave examples of the common bad practices in commercial ebook markup such as using attributes to create elements that already exist and adding tables as images rather than using html. These habits make the book less accessible to services such as text-to-speech and enhancements for the visually impaired, but also limit its discoverability on the web. Although most reading devices do not support the semantic markup possibilities of EPUB3, Google does. Machines are already using the full markup to “understand” content for better search, indexing, and creating snippets.

So while the main objective of Digital Aladore is to create a good reading edition of Aladore for users of e-ink devices, I also hope to create a flexible and reusable text that efficiently utilizes standards to be more human and machine readable. This will ensure the ebook is usable (and enjoyable) for a wide variety of current and future readers.