Skip to main content

Excel is my nemesis

Seriously, when is Microsoft going to make a setting that says, "Do NOT auto-detect data types!" so I can check that box?

I deal with raw data a lot, and we end up using CSV simply because most everyone has a program that allows for easy viewing of the data inside (I still wish people had settled on a simple delimited format, but that's another rant). Excel (2007/2010) ends up being the most common data-viewing-tool that people use.  It's unfortunate. Most people have no idea that Excel attempts to auto-detect the type of data in a field, and even less know that some of it's guesses are irreversible.

So if you send them a csv and they wish to change one little thing before sending it along, they have no idea that they just screwed up the file! Date information is the most annoying. If you send "yyyy-MM-dd HH:mm:ss", it gets transformed into "M/d/yyyy HH:mm"---not only for viewing, but it gets saved back into the CSV in that format!


So start with a CSV file with these contents:
2007-03-24 13:23:00,25E2,just some text
2011-05-01 02:11:05,11F7,more text
If you open that in Excel, adjust the columns so you can see the data, then close the file, Excel asks you to save the "changes". So if you do, and keep it CSV, your file will get changed to this:
3/24/2007 13:23,2.50E+03,just some text
5/1/2011 2:11,11F7,more text
How "fun". Note that you could work around the date format by forcing a custom format on those fields. However, the "25E2" text is forever mangled; I know of no workaround.
Labels:

Comments

Post a Comment

Popular posts from this blog

SqlBulkCopy and the "colid" error

I thought there was a page explaining this somewhere out there on the Internet, but I can't find it anymore. So here's what I re-discovered. When you try to insert the rows from a DataTable and the data in one of the columns of one of the rows is too big to fit into the destination column in the database, you get a SqlException with this error message: "Received an invalid column length from the bcp client for colid N." (Where "N" is a number.) It doesn't tell you which row, and it's a pain to figure out what column to look at. To determine what column it is referring to, you first need to get a listing of all columns in the table, listed in the order as they have been defined in the database. Next, you remove any columns in the list that are not represented in SqlBulkCopy.ColumnMappings (the order of the column mappings is irrelevant). The list that remains is what "colid" is referring to, with the first column corresponding to colid &
Post a Comment
Read more

Live Migration between domains

For those of you like me who aren't experts at all things Active Directory (AD) and Hyper-V Live Migration (LM) permissions, it can be enough of a pain to LM a Virtual Machine (VM) between domains that you simply decide to take the VMs offline to affect the move. See, I only tolerate AD because it's required for LM'ing VMs; there isn't a choice. (It's also required for Windows Clusters, but that's a different topic.) But I figured it out. My back-story is that we setup a cluster using Windows 2012 r1 as the AD Domain Controller (DC) and Hyper-V Server 2012 r1 for the VM hosts. Then we decided we wanted to use r2 for the AD DC and Hyper-V hosts. Upgrading Hyper-V was easy. But I found that there's some unresolved Microsoft bug with Windows Clustering when upgrading the AD DC from Windows 2012 r1 to Windows 2012 r2--- clustering simply doesn't work correctly anymore . So we gave up and created a from-scratch Windows 2012 r2 AD DC then made a new cluster
Post a Comment
Read more

Hyper-V and reserving RAM for the host/root/parent partition

After a long process, I finally have a real-world calculation for determining how much RAM to reserve for a Hyper-V host. The question/answer about it is here. But the summary is that Hyper-V loses RAM to the Nonpaged pool (and all of it is "untagged") in addition to the "standard" stuff that Microsoft has documented. Be aware that I write MB/GB here, when I actually mean MiB/GiB. I feel it will be more intuitive to see the notation that Windows (incorrectly) uses. Host Overhead 300 MB for the Hypervisor services 512 MB for the Host OS (This is a recommended amount; you have some wiggle-room with this.) [The amount of physical RAM available to the host OS] multiplied by 0.0425 (result in GB ) for the Nonpaged pool (Which means multiply that by 1024 to convert to "MB") Per-VM Overhead 24 MB for the VM 8 MB for each 1 GB of RAM allocated to the VM. Examples 12 GB RAM, 1 VM @2 GB, 1 VM @4 GB Host: 812 + (0.0425 * 12 * 1024) = 1,334.24 MB
Post a Comment
Read more