by Robert A. Muenchen Here is my latest update to The Popularity of Data Analysis Software. To save you the trouble of reading all 25 pages of that article, the new section is below. The two most interesting nuggets it contains are: … Continue reading →
Example 2014.10: Panel by a continuous variable
In Example 8.40, side-by-side histograms, we showed how to generate histograms for some continuous variable, for each level of a categorical variable in a data set. An anonymous reader asked how we would do this if both the variables were continuous. …
Example 2014.9: Rolling averages. Also: Second Edition is shipping!
As of today, the second edition of “SAS and R: Data Management, Statistical Analysis, and Graphics” is shipping from CRC Press, Amazon, and other booksellers. There are lots of additional examples from this blog, new organization, and other features …
Support the Blog Author to Bike MS
My dear readers, I’m asking you to support me in Bike MS (Multiple Sclerosis) and I will ride 60 miles to support the campaign in New Bern, NC, September 6 – 7, 2014. As my understanding, one of the purposes of this fundraising, is to let the message spread. MS is a very rare disease. […]
Translate SAS’s sas7bdat format to SQLite and Pandas
Thanks Jared Hobbs’ sas7bdat package, Python can read SAS’s data sets quickly and precisely. And it will be great to have a few extension functions to enhance this package with SQLite and Pandas.
The good things to transfer SAS libraries to SQLite:
- Size reduction:
SAS’s sas7bdat format is verbose. So far successfully loaded 40GB SAS data to SQLite with 85% reduction of disk usage. - Save the cost to buy SAS/ACCESS
SAS/ACCESS costs around $8,000 a year for a server, while SQLite is accessible for most common softwares.
The good things to transfer SAS data set to Pandas:
- Pandas’ powerful Excel interface:
Write very large Excel file quickly as long as memory can hold data. - Validation of statistics
Pandas works well withstatsmodelsandscikit-learn. Easy to validate SAS’s outputs.
How I Got Started with Computers
This is not your typical tech story. I trace my life as a programmer back to ninth grade, not to a computer class or even a math class, but to English. In ninth grade, I was fortunate to have English with Miss Burke who, along with the standard Shakespeare and expository writing, slipped in material […]