Category Archives: Editorial

Excel corrupts data in long numbers

I thought I was going to get some work done today, but instead I spent a significant amount of time dealing with a data corruption problem in Excel.

I had a number of files (e.g. over 250) that were all *.csv files downloaded from a server and full of useful data. Among the data is a 13-digit account number. But when I opened my files I discovered that all the account numbers had the last six digits replaced by zeros. After some experimentation I worked out what was happening. Excel was first converting all of these numbers to what is called Scientific Notation. That is what you see when a long number turns into something like 2.690565 +E11. You can change the format of the cells (to number) to make this temporarily go away, and if you do so before you attempt to save the file, you are OK. But as soon as you open the file again everything goes back to Scientific Notation again. Now, the reason that this is a Bad Thing is that if you do anything that triggers a save of the file while it is in this state it will throw away some of your data. So for example with a number like 2690565134729, Excel first converts it to 2.690565 +E11, and when you save it, Excel throws away the digits that weren’t displayed. So when you reopen your file, the number is now 2690565000000. This is a mess. And once the data is gone, it is gone and unrecoverable.

It turns out that this a known problem with Excel affecting lots of people.If you do a web search on “Disable Scientific Notation in Excel” you will find lots of people with this problem, and also that you cannot turn off this behavior. There were some suggestions of convoluted scripts or macros you could use that might help, but basically Microsoft designed it to work this way and is not interested in changing it.

Years ago I worked with a fellow who had a number of Microsoft certifications. When I complained that apparently Microsoft never did any usability testing, he quickly corrected me. As he explained it, Microsoft does extensive usability testing. They just never let it affect the design of their products.

Well, I was using Excel because that is what they give me at work. But after losing two months worth of data to this stuff, I opened up my copy of OpenOffice.org, which I keep on a thumb drive, and was happy to discover that I had no problems at all. OpenOffice.org hs no compulsion to convert numbers to Scientific Notation, does not corrupt data, and as a result I feel much safer. I configured my workstation to use OpenOffice.org by default for all *.csv files from now on.

 Save as PDF

Office 2007 Ribbon

As you may know if you have purchased a new(er) copy of Microsoft Office, they introduced a new User Interface (UI) in Office 2007 called the Ribbon, and they mean it. They make no provision for going back to the menus of Office 2003.

This has lead to some controversy, many heated exchanges, and not a little name-calling. People used to the older interface accuse the designers of making change just for the sake of making change, and the designers in turn think that the people who want the old interface are stuck-in-the-mud Luddites, who cannot keep up with needed improvements. I don’t want to weigh in on this aspect of the controversy, other than to say that I can understand where each side is coming from to some degree. I think you get this any time you attempt to change an interface element. I am seeing it now in Ubuntu with the decision to move the window buttons from the right to the left.

The point that you might want to keep in mind here is that the change gives every appearance of being permanent, and I have heard that OpenOffice.org is considering making similar changes to their UI. So if you want to simply get on with it, to get some work done without fighting any major battles, I think there is merit in learning the new UI and making peace with it. And I can offer an interesting resource to help. One of the major designers maintains a blog on MSDN, and explains a lot of this. Unfortunately, finding things on that blog is not as easy as one might hope, so someone else published a guide with links to all of the pertinent posts. That guide is available here. By going here and reading the posts that Patrick Schmid has organized (from the blog of Jensen Harris), you can start with why they thought a change was necessary, learn about the philosophy of the new Ribbon UI, and details about how it works. This is highly recommended for getting up to speed.

 Save as PDF

Windows 7 needs new hardware (sigh)

Well, it looks like Windows 7 will not work with my trusty Plextor DVD burner. I’ve tried changing drivers, updating firmware, and no matter what, Windows 7 won’t recognize the contents of the drive. It keeps telling me that there is no disc in the drive. So I think I am going to have to make a purchase to get this going. I’ve heard that Plextor may not be as good as it once was, so I am going to look into some alternative brands this time. I didn’t want to do this, but I am not really surprised either. Windows 7 is leaving a lot of stuff behind, it looks like all of my other hardware is OK. Fortunately DVD burners are not that expensive these days.

 Save as PDF

Is Office Software Obsolete?

One of the audio podcasts I listen to is This Week In Google, which despite the name actually covers everything cloud-related, not just Google. It is a very interesting podcast, with Gina Trapani, Jeff Jarvis, and Leo Laporte. In this particular episode, Leo asked the others what they used for most of their writing. In each case, it was some kind of online application, not a conventional word processor like Microsoft Word, WordPerfect, or OpenOffice.org Writer. Jeff Jarvis did allow that he had to use Microsoft word when he wrote the book, but he did phrase it as he “had to use it.” It really did sound as if he would have preferred to use something else. Gina Trapani is writing a book right now, on Google Wave, and is doing it all in a wiki that is open to others.

I think that for some things there is no real replacement for the power of a desktop program. As I mentioned in a previous post, I had a contract once at a large legal practice, and I can assure you that lawyers will not be using Google Docs any time soon to write their legal papers. And there are very real questions about security if your documents are online. Plus, you have to consider that having things online can present problems if anything happens to your Internet access or the remote servers. You would be cut off from access to your files.

A lot of pundits have taken these objections as evidence that online software will “never” replace desktop programs. I am not sure. I think there are other facts that these pundits overlook. The first is that online software will continue to be developed. The feature list of desktop software is so rich now that there really isn’t a lot left for them to do. The major new feature of Office 2007, for example, was the Ribbon, and that has received more criticism than praise as far as I can tell. The other major change is in file formats, which is not something most users care about at all. So I think online software can catch up by virtue of the fact that desktop software is pretty much at a dead end.

The second point that we need to consider is how technology will change in the next few years. Most folks have heard about Moore’s Law, which is about how processors will double in capability every 18 months or so. In fact, this type of exponential growth is not restricted to just processors. All technology is growing exponentially. Each new generation of wireless is approximately 10 times faster than the previous generation. Broadband speeds are growing, and so on. Technical barriers to “cloud” software that we see today we possibly won’t see in 2-3 years as the technology improves.

The final point to consider is that these changes often happen rather quickly when a tipping point is reached. There is a technical term, hysteresis, which we could use here. What this means is that a smooth change in the technical and economic parameters can appear to have no effect for a period of time, and then in a fairly short time frame everything changes. I suspect the change from desktop office software to online “cloud” equivalents will probably display this type of behavior. Clearly the online software has an economic advantage, since most of it is free. And online does have advantages in a world where increasingly people want to have access to all of their data all of the time, no matter where they are. We may well see a sudden shift when everything lines up to make the online applications preferable.

One last point is that this will not mean that desktop office software disappears, only that it will not be the dominant way documents, spreadsheets, and presentations are created.

 Save as PDF

Word Processors can be different

I had a contract once that involved a very large law office. The thing about law offices is that documents are, to a great extent, their stock-in-trade. Legal practice can be very productively analysed as an assembly line for the production of documents. So technologies designed to help manage and produce documents are at the heart of the business. In the dim mists of time, this meant large numbers of secretaries to type up documents, massive file cabinets to hold them, and file clerks to store and retrieve these paper documents. In the 21st century (or as close to it as legal practices can manage) these functions have been mostly taken over by computerized systems. There are still secretaries, but not as many, because many lawyers are finding that word processing technology lets them create documents directly on the computer more efficiently than trying to write out instructions for a secretary to type up. And in place of the file clerks and banks of file cabinets, there are document management systems.

This large practice had purchased such a document management system, and then discovered that it would not work properly with their standard for word processing, WordPerfect. They decided, not without howls of protest, to convert to Microsoft Word, and I was brought in to help with that transition. Trust me, the howls of protest did not stop after the decision was made. There are three groups that epitomize passive-aggressive resistance to change: doctors, lawyers, and college professors. I have worked with all of them, and I don’t think I could pick any one group as worse than the rest. They are all horrible to deal with in their own unique way, and one trait they have in common is the conviction that regardless of the topic at hand, they know more than you do. In fact, they are convinced they are generally more intelligent than you, and are somewhat dismayed that they even have to spend time talking to you.

Nonetheless the conversion proceeded. The biggest howl from all of them was “I cannot find “reveal codes”? I need reveal codes. How can I edit without reveal codes?” And the answer, which they did not like, was “There is no “reveal codes” in Microsoft Word. It does not work that way. Learn to use Microsoft Word in its own terms.” Now with the secretaries this went fairly well. Many of them were even happy to discover things they could do in Word that they could not do so easily in WordPerfect. And I was able to bring around at least a few of the lawyers to a grudging acceptance that Word could in fact do the job they needed.

The lesson here is that 90% of what a word processor does, every word processor does, and in pretty much the same way. If you want to make something bold, or change the font, save document, open a document, etc., you are going to be doing all of these things the same way no matter what program you choose. In some cases, how you do something will be slightly different from one program to the next, but you can generally get the job done if you ask the right question. In this case, instead of looking for “reveal codes” the real question is “How do I find the things that are affecting how my text displays?” And there are excellent tools for doing that in Word.

 Save as PDF