Fun News Error

From the Radio New Zealand News site this morning:

Independent testing for formaldehyde on a range of clothes has shown undetectable levels of the chemical.

Accessing Windows PowerShell from SAS

Windows PowerShell, previously Microsoft Shell or MSH (codenamed Monad) is an extensible command line interface (CLI) shell and scripting language product developed by Microsoft.—Wikipedia

PowerShell provides ‘easy’ access to Windows via a command line, and can be used for many things. I wanted to find the total disk space and amount free which we can do like this:

get-wmiobject -class win32_logicaldisk

Outputs:

DeviceID     : A:
DriveType    : 2
ProviderName :
FreeSpace    :
Size         :
VolumeName   :

DeviceID     : C:
DeviceID     : C:
DriveType    : 3
ProviderName :
FreeSpace    : 221132865536
Size         : 293552017408
VolumeName   :

PowerShell uses STDIN and STDOUT, which is how we can pipe it into SAS. Note adding the filter to restrict to local, non-removable, drives:

filename diskinfo pipe "powershell get-wmiobject -class win32_logicaldisk -filter 'DriveType=3'";
 
data _null_;
 infile diskinfo;
 input;
 put _infile_;
run;

Which yields (note the 2 leading blank lines):

DeviceID     : C:
DriveType    : 3
ProviderName :
FreeSpace    : 221132881920
Size         : 293552017408
VolumeName   :

PowerShell can reformat output, but I coded SAS to read it as-is:

data _null_;
 infile diskinfo firstobs=3 truncover;
 input
     @':' +1 DeviceID    $1.
  /
  /
  /  @':' +1 FreeSpace   32.
  /  @':' +1 Size        32.
  /  @':' +1 VolumeName $32.
  /;

 put _all_;
run;

Which gives us variables holding the total size of the C drive (in bytes) and free space:

DeviceID=C FreeSpace=221132861440 Size=293552017408 VolumeName=  _ERROR_=0 _N_=1

Being a SAS Data Warehouse Administrator

Somebody asked me what made a SAS data warehouse programmer different from a normal SAS programmer—was it the amount of SAS knowledge I had?

The short answer was no, it’s all about experience. Here’s a rough longer answer based on my data warehousing experience at 4 different organisations:

• Strong base SAS and macro is essential, this is what the majority of data warehouse jobs use.


• Also important is a strong knowledge of how SAS works internally. What gets written to the log in the following sample, and why?
  

  data _null_;
   log 'hi';
   set work.test(obs=1);
  run;

• A general knowledge of SAS and the ability to learn more as needed: e.g. I also use ODS and Graph on a regular basis.


• General knowledge: operating system knowledge (mainframe/Unix/Windows), SQL, reading from different formats (CSV, XML…), writing to various formats (Excel, HTML, XML…), version control, backup…


• Awareness of machine resources and scheduling


• Understand the big picture: can you justify creating an index (space and time vs. potential usage & savings)? Should you create another data set for new data or combine with an existing one? When should you create a macro for a common routine vs. a pre-summarised data set?


• Consistency: users need to be able to transfer their knowledge from one part of the warehouse to another. Don’t make it hard for them.


• Work within the existing warehouse setup: if the naming standards are CamelCase (my preference), don’t create Underscore_Names.


• It’s about doing it right


• Documentation. The ‘d’ word! I try to create clear and commented code, easy-to-use data sets, and document what I’ve done. My goal is to make myself redundant (which hasn’t worked!)


• Communication: essential within your team, to users, and management. Sometimes users and/or managers will have an exact requirement but it should be implemented in a different way to fit the overall picture—you need to spot these and be able to explain why.


• After hours work: users don’t like losing the warehouse to maintenance during working hours!

Bridge to Nowhere^h^h^h Terabithia

You have not stumbled into Narnia—don’t believe the trailer.

This is a coming of age movie, and as that succeeds well. However it is NOT a fantasy movie as portrayed in the trailer: the fantasy elements only make up about 10 minutes of movie time.

Take your pre-teens, not your younger kids looking for a good fantasy.

Alice’s Adventures in iPodland

A couple of years ago I wrote a SAS program to convert a Gutenberg text of Alice’s Adventures in Wonderland to an iPod eBook (iStory).

The SAS program takes the Gutenberg text file, combines hard lines into paragraphs, extracts the chapter numbers and names, and splits the whole lot into pages by chapter. It then generates a preface, table of contents, and puts Prev/Contents/Next on all the pages.

I was wondering what to do with it tonight and found a site which specialises in converting these texts to various formats: manybooks.net. Their Alice text is a lot cruder than mine—it's just a set of pages with Next/Prev—but it gets the job done and there’s a whole library there.

So here’s mine for posterity. The concept’s been superseded but it was a good learning exercise in how iPod Notes work. You can read how to install it and more about Notes at MAKE.

Download the nicely formatted version of Alice’s Adventures in Wonderland.zip

Minimum DVD Burn Speeds

I needed to burn a DVD at a slow speed for a project I’m working on—the DVD will be used as the master for some 3,500 copies so quality is important. The factory said the slower the burn the better the recording, that you can actually see the difference on the DVD surface when burning at higher speeds, and that a DVD burnt at high speeds may work OK in some computers but malfunction in others. They suggested 1× but we’ve compromised on 4×.

The reason for the compromise is the drive in my MacBook Pro won’t go any slower. Here’s an interesting matrix of drives, DVD-R medium, and the resulting available burn speeds, put together by a colleague (thanks Glenn).

Table of Drives vs. Media = Burn Speeds

Drive	Apple 4×	Transonic 8×	Verbatim 8×	Imation 16×
HL-DT-ST GWA-4080	4	4	4, 6	4, 6
HL-DT-ST GWA-4165	2, 4	4, 8	4, 8	4, 8, 16
Matshita UJ-815	1, 2	1, 2	1, 2	1, 2
Matshita UJ-825	1, 2, 4	1, 2	1, 2, 4	1, 2
Matshita UJ-835	1, 2, 4	1, 2	1, 2, 4, 8	1, 2
Matshita UJ-846	1, 2, 4	1, 2	1, 2, 4, 8	1, 2
Pioneer 104	1, 2	1, 2	1, 2	1, 2
Pioneer 106				1, 2, 4
Pioneer 109	1, 2, 4	4	4, 6, 8	4, 6, 8, 12, 16
Sony DW-U10A	1, 2, 4	1, 2, 4	1, 2, 4	1, 2, 4

All tests were done using Disk Utility in OS X 10.4 (Finder and hdiutil will give the same result).

My MBP has an HL-DT-ST GWA4080MA DVD drive, so the minimum burn speed is 4×.

(You can see the drive mechanism in System Profiler. Apple can (and does) change mechanisms within a product line.)

I Wouldn't Steal a Car, Now FOAD

A number of the rental DVDs we’re taking home are now coming with that obnoxious ‘You wouldn’t steal a car’ ad which can’t be skipped.

It’s infuriating to have some movie company override my ability to control a disc I have bought or rented. It’s offensive to have an ad before the menu accusing me of piracy.

If I download a movie from a file-sharing network (I don’t usually) there’s no copy protection and no ads.

Interesting choice: legit & annoying or illegal & convenient. I find myself buying and renting DVDs despite what the distributors are doing—my sense of right wins out over their annoying tactics.

Movie companies are pissing off their remaining legitimate customers in their war against pirates. They really don’t have a clue.

Other irritating ‘features’ on many DVDs:

Any ad (e.g. company logos, trailers, FBI warnings) preceding the menu: most of the time I just want to pop in a DVD and start watching. Include trailers and small print as a menu item.

Any menu which takes more than a couple of seconds to run through before I can push ‘play’—again, I usually just want to start watching.

Yellow Triangle

Robert Schofield of Wellington was recently convicted of importing DVDs involving urophilia into New Zealand (seemingly for his own use).

11 DVDs seems like a lot, and the guy seems to have exacerbated his fine (and maybe the new’s interest) by fighting the court and censorship office, but…

1. Likes and dislikes aside, why is the state involved in what consenting adults do in the bedroom? Why does Judge Harrop think it necessary Robert’s clients know his sexual predilections?

2. Why is the act itself legal, but depiction of it isn’t?

(Title being a play on the colour of urine and the song by Christy Moore on speaking up for your rights.)

FileMaker+QuickTime+Windows=Argh!

Normally I enjoy working with FileMaker, but on a recent project struck a number of serious issues with QuickTime container fields in a database.

Crash with Audio File

If you insert a QuickTime music file into a container field (or have a calculated container field pointing to an external file), then FileMaker will crash. Sometimes when you insert the file, sometimes when you start playing, and sometimes when you stop—but consistently crashing.

Error signature: AppName: filemaker pro advanced.exe AppVer: 8.5.2.378 ModName: quicktime.qts ModVer: 7.1.5.120 Offset: 00057ff6

We experimented with .avi files, .mp3 files, .mov files, and more. The only consistent factor was every file without a video track would crash, and every file with a video track wouldn’t.

It was hard to believe there was such a fundamental problem. We took one of the .mp3 files which was crashing and wrapped it inside a .mov with a video track (a single frame): no crash. We disabled the video track: crash. We removed the video track: crash.

Workaround: add a video track or text track. The movie controller can be resized and the border made invisible such that you can’t tell a video track is present.

A longish session with Google told us this problem had been around a long time, yet FileMaker Inc. said they’d never heard of it.

Special thanks to Allan for quickly putting a set of test files together; also Brendon, Tim @ FileMaker Inc Au, Ivan, and Patrick for their thoughts.

QuickTime Controller Jumps Ⅰ

If you have the status area showing and start playing a movie the QuickTime controller will jump to the left.

Speculation: somebody forgot to include the width of the status area when sending the ‘place to draw the controller’ to QuickTime…

Workaround: hide the status area.

QuickTime Controller Jumps Ⅱ

Set up: A layout in list mode and a QuickTime container in the header or footer.

If you start the movie playing and then scroll while the movie is still playing, the QuickTime controller will jump vertically (to a different position each time you scroll). The amount of the jump is related to the height of the body section.

No realistic workaround found 8-(

Sound Stutters

Set up: A layout in list mode and a QuickTime container in the header or footer.

If you start the movie playing and then scroll while the movie is still playing, the sound will stutter every time the layout is scrolled.

No realistic workaround found 8-(

Misc

All bugs reported… None happen on a Mac.

QuickTime 7.1.3, FileMaker Advanced 8.5v2, Windows 2000 SP2 (happens under other configurations too)

Combining Mac and Peecee FileMaker Runtimes

I recently generated a cross-platform runtime for a client who runs both Windows and OS X. Here’s some notes on the process:

1. Generate runtime on OS X
2. Generate runtime on Windows with same bindkey
3. Combine two resulting directories:
   • .USR will be duplicated (one is redundant)
   • .exe had sticky bit set resulting in ‘One or more items have special permissions and cannot be copied.’ (permissions -rwxr-Sr--, chmod g-s xxx.exe fixed it)
   • Note Windows Extensions folder has more items than Mac (so combine folder rather than replace)

I created a high level folder to hide the myriad of Windows files. So at the top level I now have:

• Solution OS X (created under OS X, alias to Solution.app)
• Solution Windows.lnk (created under Windows, link to Solution.exe)
• Documentation folder
• System Files folder
  • Extensions folder
  • Solution.app
  • Solution.exe
  • Solution.USR
  • (40 DLLs)

The client clicks on the appropriate solution name depending what platform they’re on.