tag:blogger.com,1999:blog-88851670823194910812008-07-23T23:03:25.561-04:00digital sanitation engineeringnickghttp://www.blogger.com/profile/04659163622120256748noreply@blogger.comBlogger59125tag:blogger.com,1999:blog-8885167082319491081.post-48703498650102626362008-07-10T11:20:00.004-04:002008-07-10T18:02:43.320-04:00Virtual Box 1.6.2 and Mac OS X 10.5.4: Still Fail<p>Just noticed <a href="http://virtualbox.org/">Virtual Box</a> released version 1.6.2. I tried installed Ubuntu 8.0.4 Server both 32 and 64-bit versions on my Mac OS 10.5.4 box. Depending on the Ubuntu flavor, it still either doesn't install or doesn't boot after being installed. Just saying.
</p>
<p><b>update</b>: So later in the day I decided to experiment with the <i>desktop</i> version of ubuntu.
The 64-bit version does not work, as the VirtualBox CPU detector is confused, but, the Ubuntu 8.0.4 Desktop 32-bit version <b>works</b> it installs, and boots into the desktop. Only NAT networking is implemented which means "outbound" networking is fine: web browsing, apt-get updates, etc. But you cannot ssh or copy files (directly anyways) into the virtual machine (you need bridged networking for this). The bug report is <a href="http://www.virtualbox.org/ticket/310">here</a>. Boo.
</p>nickghttp://www.blogger.com/profile/04659163622120256748noreply@blogger.comtag:blogger.com,1999:blog-8885167082319491081.post-67758617604446585242008-07-10T10:14:00.003-04:002008-07-10T10:31:45.020-04:00Varargs in Python<p>
"Varargs" is C/C++ (?) name for "variable-length argument lists" for functions and methods, used for example by <code>printf</code>. In python, this: <a href="http://docs.python.org/tut/node6.html#SECTION006730000000000000000">Arbitrary Function Arguments</a> is what you are looking for (Sections 4.7.3 and 4.74).
</p>
<p>This was written since most searches produce junk for this answer. Hopefully some web crawler will find this.</p>nickghttp://www.blogger.com/profile/04659163622120256748noreply@blogger.comtag:blogger.com,1999:blog-8885167082319491081.post-67037374108639289702008-05-06T11:00:00.002-04:002008-05-06T11:07:55.433-04:00VirtualBox 1.6 on Mac OS 10.5.2<p>
Sun's free virtualization product<a href="http://virtualbox.org/'">VirtualBox</a> released version 1.6, which supposed will fix the PAE Guest issue which prevented Ubuntu from installing last time.
</p>
<p>
The good news: The download of VirtualBox is simple and the UI is a lot better than the beta version I looked at a few weeks ago.
</p>
<p>
The bad news: There is a big warning saying "if you have a 64-bit host OS, then you need to run a 64-bit guest OS" (and the same for 32-bit). Ok, so I attempted to install Ubuntu Hardy Server 64-bit (e.g. the 'amd' version) since Leopard/10.5 is a 64-bit OS. The VM complained that it needed "x86_64" but only found "i686". Sigh. Ok, let's install Hardy Server 32-bit. It installs, but does not boot, and gives a complaint about the CPU and "0.6" (i.e. a random error message).
</p>
<p>Ok so much for that. Looks like we'll wait for version 1.7</p>nickghttp://www.blogger.com/profile/04659163622120256748noreply@blogger.comtag:blogger.com,1999:blog-8885167082319491081.post-81899094559003408132008-04-28T10:02:00.006-04:002008-04-28T10:40:51.703-04:00Virtualization is faster than Native on Mac OS X 10.5?<p>
That little article on Battle of Mac Virtualization Products caused quite a stir with Parallels and another web publisher contacting me. I'll follow up with them and report back but in the meantime check this out.
</p>
<p>I mentioned last time that on Parallels, the linux filesystem (ext3) seemed quite a bit faster than <i>the native filesystem</i>. They are many ways this could happen and honestly I have neither time not expertise to learn about the finer details of HFS3 vs. ext3. Filesystems are messy.
</p>
<p>Not that CPU benchmarks are much better, but I do have a silly python benchmark that tests raw performance. I stole the idea from <a href="http://furryland.org/~mikec/bench/">http://furryland.org/~mikec/bench/</a>. I use it mostly to see if compiler flags make any difference when compiling python from source, or what the performance of a pre-made version of python is. The code is at the end.
</p>
<h3>Apple OS X 10.5</h3>
<pre>
$ ./benchmark.py
test_hash: 2.833932
test_list: 3.647719
</pre>
<h3> Ubuntu 8.0.4 Server 64-Bit in VMWare Fusion</h3>
<pre>
$benchmark.py
test_hash: 1.697652
test_list: 2.255775
</pre>
<p>Uhhhh, say what? VMWare is 33%+ faster than native? Remember this is running the same code, on the <i>same machine</i>. I have no idea if Ubuntu being 64-bit (vs. 32) makes any difference. This code doesn't do very much, so I suspect that the Linux memory allocators must be <i>quite</i> a bit better than Mac OS X.
</p>
<p>This is of course a bit synthetic, and I can find other examples of the virtualized linux being a lot slower than native mac, however, it is interesting.</p>
<p>Comments most welcome.</p>
<hr />
<h3>The Lame Test</h3>
<pre>
from timeit import Timer
test_hash = """
for i in xrange(100):
x={}
for j in xrange(1000):
x[j]=i
x[j]
"""
test_list = """
v=['a','b','c','d','e','f','g']
for j in xrange(1000):
v.append(j)
v[j]
"""
t = Timer(stmt=test_hash)
print "test_hash: %f" % t.timeit(100)
t = Timer(stmt=test_list)
print "test_list: %f" % t.timeit(10000)
</pre>nickghttp://www.blogger.com/profile/04659163622120256748noreply@blogger.comtag:blogger.com,1999:blog-8885167082319491081.post-50639647850254764712008-04-25T13:44:00.002-04:002008-04-25T15:37:38.535-04:00Battle of Mac Virtualization Products<p>
I just evaluated three virtualization product for the Mac: <a href="http://www.parallels.com/en/products/desktop/">Parallels Desktop</a> For Mac 3.0 (build 5584), <a href="http://www.vmware.com/products/fusion/">VMware Fusion</a> 1.1.2, and <a href="http://www.virtualbox.org/">Virtual Box</a> (Version 1.5.51) to run Ubuntu Linux 8.0.4 (Hardy) <i>server</i>. (not the desktop version). My test system is a MacBook (not pro) with 2G of RAM and 2.2 Ghz Intel Core 2 Duo on Mac OS X 10.5.2 (Leopard). This is an 64-bit CPU and OS.
</p>
<p>
Ok, now that I'm done with adding keywords for search engine, you might <i>why?</i>. The goal is develop software on a platform that is as close as possible to the final deployed version. But isn't Mac with it's "BSD-core" along with MacPorts or Fink or Gentoo Linux for Mac good enough? The short answer is no, and I'll write that up in another post. You might also ask why don't I just ditch Mac altogether and just Ubuntu Desktop. Here the answer is because I like the Mac hardware, the OS, and mostly because cause I don't want to. I suppose later on that well might be a good answer, but even then, I'd still use virtualization (again, another post).
</p>
<p>
Most of these product heavily advertise on Windows-Mac integration and accelerated graphics and whatnot. I'm not testing any of that. I'm testing how easy is it to install and the performance.
</p>
<h3><a href="http://www.parallels.com/en/products/desktop/">Parallels Desktop</a> For Mac 3.0</h3>
<p>I downloaded this, got my 15-day trial key and we are off to the races.</p>
<p>The UI is pretty good, and the installed answering a few questions wasn't bad, although it wasn't always very clear. A few options I needed were hidden and took a few times to figure out what exactly I needed to do. To be fair, it was my first time using <i>any</i> virtualization product.</p>
<p>Ok the bad news is that Parallel needs a bit of help to get the server install to boot. The installer kernel is fine, but the stock server kernel does not boot. You need to then "Repair broken install" drop down to root, uninstall the kernel and then install th stock desktop kernel. (Read about it: <a href="http://snippets.aktagon.com/snippets/101-Solution-for-Kernel-panic-CPU-too-old-for-this-kernel-when-installing-Ubuntu-on-Parallels">here</a>, <a href="https://bugs.launchpad.net/ubuntu/+source/linux-source-2.6.22/+bug/151942">here</a>, and <a href="http://virtualbox.org/ticket/212">here</a>)
</p>
<p>64-bit Ubuntu kernels did not boot either. I didn't try the hack to swap kernel versions</p>
<p>Anyways, once it's up, it works, and works well. Networking is no problem, ifconfig works, you can find your IP and ssh into the box, no problem. I did find occasionally if I put the computer to sleep, the networking didn't always return. Doing "nothing" (waiting for login prompt) the Parallels VM takes 12-16% of the CPU. If you suspend the VM, 3% of CPU is being used (for what?). That said, over all the performance is very good. For a few of the processes I tested, the overall runtime was about equal to native, but much more CPU was used (which is fine).
</p>
<p>Ok here's one crazy thing. The file system seems faster on the VM than on the native mac osx! (there are many ways this could be possible but who can say).
<h3><a href="http://www.virtualbox.org/">Virtual Box</a> 1.5.51 Beta 3</h3>
<p>Their big claim to fame is that it's open source and free. These guys just got bought by Sun. It's not clear what they are intending to do with them. But you can guess. It's a GPL VM so any Linux distro can include it. This will make OpenSolaris easy to install and try out. And you can guess that OpenSolaris will run the best. Who can say. Anyways...
</p>
<p>
Sadly, at least for now on Mac, you get what you pay for. The UI is very confusing, and frequently dead-ends, where you effectively have to restart (you get a modal dialog with no buttons... nice). While the install kernel booted the final image did not (this appears to be fixed in an <a href="http://virtualbox.org/ticket/212">upcoming version</a>) Also compared Parallels, it was noticeably slower. The idle VM took 25% of the CPU (!!) during the install.
</p>
<p>Given the price point it's worth checking out, and the version I checked was a "beta 3". We'll see what Sun does.</p>
<h3><a href="http://www.vmware.com/products/fusion/">VMware Fusion</a> 1.1.2</h3>
<p>
I spoke with a VC yesterday and we had a nice chat on virtualization on Mac and recommended <a href="http://www.vmware.com/products/fusion/">VMware Fusion</a>.
</p>
<p>Downloaded, 30-day trial key (retail is $80 same as Parallels), and..</p>
<p>Whoa. So litterally in about 5 seconds of starting this up, my Ubuntu iso image was booting and installing.
</p>
<p>The final install booted without any monkey business like Parallels and VirtualBox</p>
<p> VMWare correctly emulated, and I <i>did not</i> have to use
<p>The idle VM takes on 5%, and when suspending VMWare takes 0% CPU. I like that!</p>
<p>And without any modifications, VMware can install and boot 64-bit kernels (i.e. ubuntu-8.04-server-amd64.iso)</p>
<p>And as a bonus, VMware provide pre-configured VMs with various linux distributions (Fedora, Cent, etc).</p>
<h3>Conclusion</h3>
<p>
So right <i>now</i>, 25-Apr-2008, VMware is the clear winner. But honestly, the VM space is hyper-competitive. In a few months, this will probably be wrong. Parallels certainly works and works well. If the VirtualBox cleans up it's act, the whole basic server VM space on Mac might be commodity.
</p>nickghttp://www.blogger.com/profile/04659163622120256748noreply@blogger.comtag:blogger.com,1999:blog-8885167082319491081.post-64461173665412820412008-04-09T18:58:00.007-04:002008-04-09T19:22:20.380-04:00Getting Started with EC2<p>
Wow is EC2 fussy. You know this already. Such is life when using public-key encryption. if you are just getting started, the following greatly simplifies the situation. It won't handle multiple accounts or multiple instances, but sometimes one is all you need.
</p>
<p>
First, go through the stock EC2 tutorial. And then compile all the keys and stuff into one spot like so:
</p>
<pre>
export S3_PUBLIC='your s3 public key'
export S3_PRIVATE='your s3/private key'
export EC2_PRIVATE_KEY='./pk-YOURCERT.pem' # somewhere
export EC2_CERT='./cert-YOURCERT.pem' # somewhere
export EC2_HOME='./ec2-api-tools-1.3-19403' # OR SOMETHING
export PATH=$EC2_HOME/bin:$PATH
export JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Versions/1.5.0/home
export AMI='ami-226e8b4b' # or something
export EC2_SSH_KEY=yourname-keypair
</pre>
<p>
Ok, now the following functions make it simple to start stop and login
</p>
<pre>
# VERSION 3 -- now with ec2do
# VERSION 2 -- now with push/pull
function ec2start {
ec2-run-instances $AMI -k $EC2_SSH_KEY
}
function ec2list {
ec2-describe-instances
}
function ec2host {
# it's one line below
ec2-describe-instances | grep INSTANCE | grep amazonaws.com | head -n 1 | awk '{print $4}'
}
function ec2id {
# it's one line below
ec2-describe-instances | grep INSTANCE | grep amazonaws.com | head -n 1 | awk '{print $2}'
}
function ec2stop {
echo "Stopping..."
ec2-terminate-instances `ec2id`
}
function ec2login {
ssh -i id_rsa-$EC2_SSH_KEY root@`ec2host`
}
function ec2push {
scp -i id_rsa-$EC2_SSH_KEY $1 root@`ec2host`:~/
}
function ec2pull {
scp -i id_rsa-$EC2_SSH_KEY root@`ec2host`:$1 .
}
function ec2do {
ssh -i id_rsa-$EC2_SSH_KEY root@`ec2host` $@
}
function ec2help {
echo "haha"
echo ""
echo "ec2list -- alias for ec2-describe-instances"
echo "ec2start -- starts one instance"
echo "ec2stop -- stops an instance"
echo "ec2id -- lists the reservation id of a running instance"
echo "ec2host -- lists the host of a running instance"
echo "ec2login -- log in to running instance"
echo "ec2push localfile -- push a file to remote homedir"
echo "ec2pull remote -- pull a remote file to local current dir"
echo "ec2do cmds -- execute a command remotely"
}
</pre>
<p>
Put all of that into your profile, and you'll be all set. Enjoy..
</p>nickghttp://www.blogger.com/profile/04659163622120256748noreply@blogger.comtag:blogger.com,1999:blog-8885167082319491081.post-56669211928559968382008-04-07T18:26:00.004-04:002008-04-07T18:45:46.515-04:00Python and JSON performance<p>
Ola. So I was working on a project and noticed that json encoding/decoding of a string to string dictionary was taking... uhhh 70% of the time. The are a few other benchmarks out there (<a href="http://kbyanc.blogspot.com/2007/07/python-serializer-benchmarks.html">kbyanc</a>, <a href="http://blog.hill-street.net/?p=7">hill-street</a>), but none including the new simplejson 1.8.1 which has extensions to improve performance (links below).
</p>
<h3>My Tests</h3>
<p>
I'm only serializing dictionary mapping strings to strings. Maybe occasionally dict within a dict, but always primitive types. I don't need special ways of serializing custom objects. This test is just raw primitive types and data structure performance.
</p>
<p> For this test, I made two dictionary with a dozen entries. One has both keys and value as ascii regular python strings, the other had the same data but used unicode strings.
</p>
<h3> cjson 1.0.6 </h3>
<a href="http://www.vazor.com/cjson.html">cjson 1.0.6</a>, which has the bug fix described <a href="http://blog.extracheese.org/2007/07/when-json-isnt-json.html">here</a> and the original source and homepage is <a href="http://python.cx.hu/python-cjson/">over here</a>. It's bit confusing, but it's worth it:
<pre>
Serialization, cjson, ascii: 94
Serialization, cjson, unicode: 117
Deserialization, cjson, ascii: 80
</pre>
<h3> simplejson 1.8.1</h3>
<p> Available <a href="http://pypi.python.org/pypi/simplejson/1.8">here</a> The author maintains a <a href="http://bob.pythonmac.org/archives/category/python/simplejson/
">blog</a>.
<p>
This version has c-extensions for both encode and decode
</p>
<pre>
Serialization, simplejson, ascii: 549
Serialization, simplejson, unicode: 592
Deserialization, simplejson, ascii: 2068
</pre>
<h3> simplejson 1.6 </h3>
<p>
This version is 100% pure python. It can be grabbed <a href="http://pypi.python.org/pypi/simplejson/1.6">here</a>
</p>
<pre>
Serialization, simplejson, ascii: 2150
Serialization, simplejson, unicode: 1744
Deserialization, simplejson, ascii: 3318
</pre>
<h3> Conclusion </h3>
<p>Well, guess which version I'll be using. simplejson has all sorts of other features that you might want. Sometimes speed isn't everything, but for my application it is. Enjoy!</p>nickghttp://www.blogger.com/profile/04659163622120256748noreply@blogger.comtag:blogger.com,1999:blog-8885167082319491081.post-30180764745237418542008-04-01T17:39:00.005-04:002008-04-01T18:24:52.524-04:00Hannah Montana and other innovations in shopping<p>
Over New Years I was up visiting my friend Greg and his family. His dad, Jim, is the distributor of the Hannah Montana video game (sales were so good, Jim says "it's the reason we have a Christmas this year"). But even within the household, it was a big hit with the ladies. It's a Wii game, so using the controllers you do a Guitar Hero type of thing except instead of music is dance moves. No one cared about that. But it earned points to <i>go shopping</i> in <i>virtual stores</i> where you can <i>try stuff on</i>. It's done quite well, and less retarded (hmmm, or maybe <i>more</i> retarded) than I made it sound.
</p>
<p>
I'm always looking for the Next Big Opportunity. Oddly, even though the Wii has the Interwebs built-in, there is no product placement and no way to add more items to buy or share outfits with the other Mii (your virtual self icon). Being a complete dork, and while watching Cheryl and Emily <i>freak out</i> on this game ("No, go back and get the other dress." "oh this is cute", etc...), and thinking, "you can put this on the Web and make MILLIONS".
</p>
<p>Surprise, I have decided not to launch a consumer shopping website targeting women, but I keep an eye on the space. Since then, the <a href="http://www.hellokittyonline.com/us/">Hello Kitty</a> online world is about to launch, <a href="http://www.glammedia.com/">Glam Media</a> (ads for women) is rocking, and yahoo launches <a href="http://omg.yahoo.com/">OMG</a> and <a href="http://shine.yahoo.com/">Shine</a>.
</p>
<p>But the most interesting is the oddly named <a href="http://www.polyvore.com/cgi/home">polyvore.com</a>. This going to be huge. It's part art-project, part paper-dolls, part social-network, but using <i>real items</i> from existing web sites. Instead of me explaining, just go to the site and click "explore". So far a few of my female friends are completely <i>addicted</i> to it. In addition to obvious CPA deals, they also white label their tech to web sites to provide virtual dressing rooms. There are other versions of this <a href="http://www.stardoll.com/en/">Stardoll</a> and <a href="http://home.zwinky.com/">Zwinky</a> for even younger kids, but this is by far the most interesting.
</p>
<p>If you know anything about shopping, and, uhh, pre-teen girls, 2008 is the time to launch websites and video games targeting them. Me? Well, I go clothes shopping like 2 times a year, in mostly non-colors. And for tech? I mostly do servers and find UI development annoying. And I don't know doodle about video game systems development. FAIL. But maybe you can think of something.
</p>nickghttp://www.blogger.com/profile/04659163622120256748noreply@blogger.comtag:blogger.com,1999:blog-8885167082319491081.post-51364864743848986282008-03-28T01:31:00.003-04:002008-03-28T01:33:21.723-04:00Quote of the Day<p> on the need for <a href="http://www.regdeveloper.co.uk/2008/03/27/google_summer_code_debian_losers/comments/"> multiple linux desktop frameworks</a>:</p>
<blockquote>
finishing a desktop [framework] is like achiving victory in Iraq,
</blockquote>
<p>
Does this mean the surge is not working?
</p>nickghttp://www.blogger.com/profile/04659163622120256748noreply@blogger.comtag:blogger.com,1999:blog-8885167082319491081.post-9187709614114712742008-03-28T00:02:00.002-04:002008-03-28T00:49:09.536-04:00Why Flash Will Never Be On The IPhone<p>
Ok, so the madness of Iphone SDK came out a week or two ago. And the big whoop-de-do was what about Flash on iphone. Never going to happen. Well it might if Adobe writes a huge check to Apple. Otherwise never.
</p>
<p>Ever wonder why the ipod does't have a <i>radio</i> in it? It would cost nothing, and have no drain on battery. Same reasons why you'll never see Flash. Apple makes $0 on radio (or flash), it strengthens the 3rd party (for radio, it improves radio advertising, for iphone it improves Adobe's developer base), and in fact competes with their technology (here radio vs. Itunes, for iphone it's flash vs. standards, a bit of an inversion but you get the idea).
</p>
<p>
But why <i>not</i> flash?
</p>
<p>
Ok <i>why</i>?
</p>
<p>Ok why use flash at all? I see a few reaons</p>
<ul>
<li>"rich media" advertisements. I'm sure AT&T is excited to use their bandwidth for this. They and Apple make $0 on from this and make the whole page slower.</li>
<lI>"vanity sites" -- the stuff that ad agencies make when a new toothpaste comes out. Not compelling and orphanware in a week. Typically completely unusable on desktop let alone the mobile browser.</li>
<li>Video and Audio. Ok everyone. <i>Think</i> how ridiculous this is. You need an abstraction layer (flash) on an abstraction layer (the browser) on an abstraction layer (the OS) to the hardware accelerated codec. More on this later.</li>
<li>Interesting animations and vector graphics. Sadly if you want a true cross platform vector graphics, flash is it. SVG isn't supported very well. YET. But the use cases for vector graphics on a web page on a mobile phone are so far low</li>
<li>Actually real applications. I will admit, the Flex API is pretty nice. I actually like it a lot. But anything you write for it will
not out of the box work on a small screen.</li>
</ul>
<p>
Ok let's go back the audio and video part. I'll spare you my rants on W3C, and just note that a week before the SDK announcement, Apple (cough, WebKit) put into mainline HTML5 support for video and audio codecs. Think of it as a restricted activex for media. You can display any number of image format, why not MP3 and Video too? Also Safari has really improved their SVG implementation. They'll pop out Safari 3.2 with improved video, audio, and svg support and then, really, <i>why</i> do you need flash on the iphone?
</p>
<p>
I don't want to be too hard on flash. I do think an integrated platform, like the one they provide, is useful. And with the state of SVG right now, they <i>are</i> the only game in town for complicated <i>desktop</i> graphics. But the iphone is different. There is no illusion that if you write something "for mobile" it will work well on all mobile phones. Plus Apple isn't going to allow alternative browsers on the Iphone. Flash does play a part here in smoothing over differences in browsers, but again, not the case here.
</p>
<p>So there you go. You'll never see flash on the iphone.</p>nickghttp://www.blogger.com/profile/04659163622120256748noreply@blogger.comtag:blogger.com,1999:blog-8885167082319491081.post-3296860954415702712008-03-10T16:36:00.002-04:002008-03-10T16:37:57.189-04:00Client9.com<p>Hello World!<p>
<p>Yes I have bought <a href="http://client9.com/">client9.com</a>!</p>
<p>There is an <a href="http://client9.com/faq.html">FAQ</a> for the curious</p>nickghttp://www.blogger.com/profile/04659163622120256748noreply@blogger.comtag:blogger.com,1999:blog-8885167082319491081.post-82381348749472877542008-03-05T18:53:00.002-05:002008-03-05T19:02:00.726-05:00Required Reading in Software Engineering, part 1<p>
Ok, that CouchDb Guy, Damien Katz, not only is writing some great code, but also writing some great articles.
This one should be required reading:<a href="http://damienkatz.net/2006/04/error_code_vs_e.html">Error codes or Exceptions? Why is Reliable Software so Hard?</a>. It's not <i>really</i> about error codes and exceptions. That's just the hook. uhhhhh, yeah, something like that anyways.
</p>nickghttp://www.blogger.com/profile/04659163622120256748noreply@blogger.comtag:blogger.com,1999:blog-8885167082319491081.post-72602817788268535492008-03-05T18:25:00.002-05:002008-03-05T18:29:41.007-05:00Recursion unsafe in C/C++?<p>
One of the guys who is writing CouchDB, wrote up this this interesting note: <a href="http://damienkatz.net/2008/02/recursion_unsaf.html">Recursion unsafe in C/C++</a>. It starts off with this
</p>
<blockquote>
The question of "why disallow recursion in C++?" came up in a comment to the C++ coding standards for the Joint Strike Fighter.
</blockquote>
<p> I was just thinking the same thing. ha!</p>nickghttp://www.blogger.com/profile/04659163622120256748noreply@blogger.comtag:blogger.com,1999:blog-8885167082319491081.post-23957545855544363652008-03-05T18:22:00.003-05:002008-03-05T18:25:21.510-05:00CouchDB<p>
FYI -- <a href="http://couchdb.org/CouchDB/CouchDBWeb.nsf/Home?OpenForm">CouchDB</a>, provides similar type of service as Amazon's <a href="http://www.amazon.com/b?ie=UTF8&node=342335011">SimpleDB</a>, including redundant copies. Check it out. The developers seems like smart cookies. I think they are speaking at <a href="http://en.oreilly.com/et2008/public/content/home">ETech2008</a>
</p>nickghttp://www.blogger.com/profile/04659163622120256748noreply@blogger.comtag:blogger.com,1999:blog-8885167082319491081.post-38038856442782607112008-02-06T21:11:00.000-05:002008-02-06T21:19:03.997-05:00Release Engineer Camp - Biz idea #3<p>
Here's another business idea. <b>Release Engineer Camp</b>. I can't tell you how many times I get asked for this or had to look for this role. it's just about impossible to hire for, and it pays better than the equiv job in say, sysadmin, ops, qa, or even php dev. So, why not have a school or camp for this?
</p>
<p>It could even be online -- just post the lessons online and hope ambitious people read up. Or you could make some thing more interactive, certificates and all that stuff. Honestly I bet you could get local area business to <i>pay or sponsor</i> this. Or sell the notes to O'Reily for a book.</p>
<p>Here's a sample plan</p>
<ul>
<li> unix basics</li>
<li> bash </li>
<li> perl or other equiv language programming</li>
<li> system admin basics </li>
<li> Source control</li>
<li> Issue tracking systems </li>
<li> Apache and equiv basics.</li>
<li> More source control</li>
</ul>
<p>Who wants to help write a chapter?</p>nickghttp://www.blogger.com/profile/04659163622120256748noreply@blogger.comtag:blogger.com,1999:blog-8885167082319491081.post-82408321300586615552008-01-27T21:58:00.000-05:002008-01-27T22:35:28.698-05:00Another project/business idea for you<p>
Previously I mentioned the need for a "liveblogging" server (and javascript client code) to handle millions of concurrent users. Perhaps not the biggest market, but in time you could extend it to other markets
</p>
<p>
Here's another idea. Recently I had the joy of playing around with ActiveMQ, the J2EE messaging api. I'm sure it's "fine"... the problem is that it doesn't have a focus. it does every possible type of messaging, pub-sub combination. And to handle this is uses the typical java bloatware. The internal engine is Derby. To store a single message they use a ful SQL database! Nutty! Also there are a million parameters, each with <i>different</i> ways of setting them. Some are in a config file, some are in the connection handler, some are in the message, some are JMX. As if the usual 1,000,000 java XML files weren't enough.
</p>
<p>On the other end of the spectrum is Amazon's Queue service. I haven't kicked the tires, but people who have say it's good for small scale applications. Interesting idea.</p>
<p>Lots of small companies would <i>love</i> to use a queue to allow them to decouple various components. Most aren't financial. So if a message happens to get lost, it's not the end of the world. most of the time it doesn't even matter that much the order (meaning if they are out of sequence it's not horrible either).
</p>
<p>So here's the idea. Queue in a Box. Not like <a href="http://tervela.com/">Tervela</a>. That's too high end. I'm talking about a $5k box, where you plug it in, Support a few protocols like memcached or STOMP or HTTP REST. A nice web interface to configure it, probably doesn't need more than dozen paramters, some monitoring. Add some raid disks for durable storage. DONE. One tricky part is to make it so you can just add more Q-in-a-Box and "it just works".
</p>
<p>I personally know a few people <i>right now</i> that would buy these. hell you could expand the product offering with Squid-in-a-box too.</p>nickghttp://www.blogger.com/profile/04659163622120256748noreply@blogger.comtag:blogger.com,1999:blog-8885167082319491081.post-36770770989440236652008-01-15T11:55:00.000-05:002008-01-15T12:59:40.994-05:00Project for U: LiveBlogging Client/Server<p>It's that time of year again. MacWorld 2008 is just about to start and already all of the liveblogging sites are clogged. Some are completely down due to demand. Others force you do hit reload, which fails 50% of the time. Why doesn't a good liveblogging client/server exist?</p>
<p>Here's what you'd need and other random notes:</p>
<ul>
<li>
A simple client library that does a poll to the server asking for everything since "last update" and updates the page appropriately (incremental). A little icon saying when next poll is, that a network connection is happing would be swell. If the "event" is over, then it knows to stop sending the request.
</li>
<li>The server is more interesting. First you need a way of publishing (writing) data and no matter how busy the server is, the publisher must be able to get in. This probably means two servers to be absolutely sure, but who knows. </li>
<li>The ability to add more front end servers</li>
<li>A ridiculously fast HTTP server. As a goal a single server should do about 10k connections per SECOND. If you poll once a minute this hopefully will be able to serve 50k people per box.</li>
<li>Event-driven http is probably the way to go since you are going to be I/O bound, not CPU bound, but you could start with anything server for kicks and swap out the server as you go.</li>
<li>You could use fixed-size message block internally, say 1-4k, to make memory or file lookups easy. (e.g. append to the file, then the clients can stat the file and read from the end and go backwards to get the updates). Or something</li>
<li>A very fast time out.. any client taking more than a second, or even less, should get the boot!</li>
<li>You get the idea</li>
</ul>
<p>It's a good project to work since you'll be able to work with all the fun stuff you normally don't get to do fool around with. It's not much of a <i>business</i> but who knows. Start with an open source and then do consulting and then hosting. If anyone is hell bent on doing this, let me know!</p>
<p>Note: <a href="http://coveritlive.com/">CoverItLive.com</a> appears to do this, but uhhh, not so well, since it's front door says it's out of capacity</p>nickghttp://www.blogger.com/profile/04659163622120256748noreply@blogger.comtag:blogger.com,1999:blog-8885167082319491081.post-83113819790308617812008-01-10T11:28:00.000-05:002008-01-10T11:48:46.509-05:00The Pitfalls of Java as a First Programming Language<p>I bumped into an interesting article <a href="http://www.stsc.hill.af.mil/CrossTalk/2008/01/0801DewarSchonberg.html">Computer Science Education: Where Are The Software Engineers of Tomorrow</a> which is really about why Java is poor <i>first</i> language to learn. (I found this by way of <a href="http://www.theinquirer.net/gb/inquirer/news/2008/01/08/boffins-deride-java">The Inquirer</a> -- lots of funny comments). The authors are professors at NYU.
</p>
<p>Juicy quotes include:</p>
<blockquote>
Java ... encourages the programmer to approach problem-solving like a plumber in a hardware store: by rummaging through a multitude of drawers (i.e. packages) we will end up finding some gadget (i.e. class) that does roughly what we want. How it does it is not interesting!
</blockquote>
<blockquote>
It [Texas A&M] did [teach Java as the first language]. Then I started teaching C++ to the electrical engineers and when the EE students started to out-program the CS students, the CS department switched to C++.
</blockquote>
<p>It's not a slam on Java, it's slam as using it as a first language, or perhaps how Java and programming are taught together. The author insist everyone must learn C, C++, Lisp, Java, and ... Ada. Huh? Well, the authors also run some Ada company. Oh well.</p>
<p>An interesting read, whether or not if you agree with their conclusions.</p>nickghttp://www.blogger.com/profile/04659163622120256748noreply@blogger.comtag:blogger.com,1999:blog-8885167082319491081.post-41607960456911849702008-01-06T20:24:00.000-05:002008-01-06T20:28:22.717-05:00stringencoders 3.7.0 released<p>Hi all,
</p>
<p>
I just released <a href="http://stringencoders.googlecode.com/">stringencoders 3.7.0</a>. This fixes autotool/libtool which was borked on some platforms according to my spies. It also fixes up some C++ header annoyances. There are no logic changes, so if there is no need to update if it has been working for you so far.</p>
<p>
<b>BUT</b>, if you haven't updated since 3.4, please do so since 3.4 has a core dumping bug, and 3.5 are some minor issues with <code>modp_dtoa</code>.
</p>
<p>enjoy</p>nickghttp://www.blogger.com/profile/04659163622120256748noreply@blogger.comtag:blogger.com,1999:blog-8885167082319491081.post-64544201392679119072008-01-05T22:42:00.000-05:002008-01-05T22:59:19.830-05:00Change Leopard Dock to 2D<p>
This is old news by now, but essential. It changes that gratuitous 3D shinny Dock to a more plain and usable 2D version. I learned about this tip <a href="http://www.silvermac.com/2007/change-leopard-dock-to-2d"/>here</a> (which also has screenshots).
</p>
<p>From Terminal, type:</p>
<pre>
defaults write com.apple.dock no-glass -boolean true && killall Dock
</pre>
<p>Ta-Da. To undo change the <code>true</code> to <code>false</code></p>nickghttp://www.blogger.com/profile/04659163622120256748noreply@blogger.comtag:blogger.com,1999:blog-8885167082319491081.post-58350857834603502542008-01-05T22:18:00.000-05:002008-01-05T22:30:46.092-05:00Making Finder be more like 'ls -a'<p>Finder has this annoying habit of hiding files and extensions it thinks will confuse you. As if!</p>
<h2>Show all files</h2>
<p>By default Finder hides files that start with "." (dot) and perhaps others too. To see all files, in Terminal type this
<pre>
defaults write com.apple.finder AppleShowAllFiles true && killall Finder
</pre>
<p>
To revert, change the <code>true</code> to <code>false</code>. I originally bumped into this <a href="http://www.jappler.com/blog/archive/2006/04/26/show-hidden-files-in-mac-os-x">here</a>.
</p>
<h2>Show all extensions</h2>
<p>Finder sometimes likes hiding the file extention like ".html" from you. To fix this: go into Finder's preferences, click the Advanced, and then click "Show all file extensions"<p>
<p>Enjoy!</p>nickghttp://www.blogger.com/profile/04659163622120256748noreply@blogger.comtag:blogger.com,1999:blog-8885167082319491081.post-64405336527442543332007-12-20T12:23:00.001-05:002007-12-20T14:08:24.497-05:00Unit testing in PHP<p>
I needed to do some unit testing for some PHP5 utility classes, and wanted some unit test framework. Here are my requirements:
</p>
<ul>
<li> Simple. If it's more than one file, something is wrong.
<li> Unrestrictive license. I should be able to include the one file in any package I create for distribution, give to clients, whatever.</li>
<li> Short namespace. I don't want to type <code>SomeBloatedClass::testIfTrueOnTruesdayOnly(...)</code>, when something like <code>assert("foo" == "bar")</code> will do.</li>
<li>Easy to run, no need for anything more than a command line</li>
</ul>
<p>Oddly I couldn't find anything. Lots of links to outdated information. Lots of monster web framework testing things. PHPUnit is baffling. I found out of date versions, links to PEAR that didn't work, etc. And it's huge! Why? Unit testing should be short and sweet.</p>
<p>So screw it: here's <a href="http://modp.com/downloads/UnitTest.php.txt">my 100-line version</a>. It does just about everything the fat ones do. Here's the "manual" (more samples in source file)</p>
<ul>
<li>Add <code>require_once('UnitTest.php');</code></li>
<li>Create a class that <code>extends UnitTest</code></li>
<li>Make a test by adding a method to the class. The method name <b>must</b> start with <code>test</code></li>
<li>Test values in the test method with <code>assert</code> and <code>assertEquals($val, $expected)</code></li>
<li>Optionally define a <code>public function setup()</code>, <code> public function teardown()</code> functions. These will be called before and after <i>each</i> test.</li>
<li>Add at the bottom <code>unittest('<i>classname</i>')</code></li>
<li>Run test with <code>php <i>yourfile</i></code></li>
<li>Script will return 0, if all tests succeeded, 1 if failures occurred, and 2 if the class couldn't even be loaded</li>
</ul>
<p> For example: </p>
<pre>
#!/usr/bin/env php
<?php
require_once('UnitTest.php');
class ATest extends UnitTest
{
public function testPass()
{
assert(1 == 1);
}
public function testNot()
{
assert(1 == 2);
}
public function testFail()
{
assertEquals("foobar", 10);
assertEquals("foobar", 10, "optional message goes here";
}
public function testError()
{
throw new Exception("wtf");
}
}
// RUN IT
unittest('ATest');
?>
</pre>
<p>Running this will produce: </p>
<pre>
$ ./sample.php
testPass: OK
testNot: Fail : Assertion failed @ line 63 in ATest::testNot
testFail: Fail : foobar != 10 @ line 63 in ATest::testFail
testError: Error : wtf @ line 24 in <i>filename</i>
Total Tests: 4
total = 4, pass = 1, fail = 2, error = 1
$ echo $?
1
</pre>
<p>
Yeah the output isn't so pretty. So... change it! Need more fancy methods? Add 'em! You can modify the base, or subclass it.
need "Test Suites" ... use directories!</p>
<p>And go forth and make unit tests!</p>
<p>Oh yeah, improvements and suggestions welcome. One more method might be <code>assertEqualsFloat</code> which would take 2 floats and a tolerance.</p>nickghttp://www.blogger.com/profile/04659163622120256748noreply@blogger.comtag:blogger.com,1999:blog-8885167082319491081.post-16334001713572501832007-12-19T20:01:00.000-05:002007-12-19T20:23:20.433-05:00Google Charts API<p>I was poking around <a href="http://code.google.com/">code.google.com</a> and bumped into the weirdest Google API yet: The <a href="http://code.google.com/apis/chart/">Google Charting API</a>. It's how <a href="http://finance.google.com/">finance.google.com</a> makes it's charts. You give 'em a URL, they give you a good looking graph png image. For instance:</p>
<pre>
<img src="http://chart.apis.google.com/chart?cht=p3&chd=s:hW&chs=250x100&chl=Hello|World" alt="achart" />
</pre>
<p>
makes this (live example):
<img src="http://chart.apis.google.com/chart?cht=p3&chd=s:hW&chs=250x100&chl=Hello|World" alt="achart" />
</p>
<p>Ta-Da. Lots of other chart types are available, too. It's good for 50k request per day. And of course you need more than that, your backend process can make the graph and copy it to your local image server.</p>
<p>What's sad is that the quality of the graphs is a lot better than most other graphing/charting solutions out there.</p>nickghttp://www.blogger.com/profile/04659163622120256748noreply@blogger.comtag:blogger.com,1999:blog-8885167082319491081.post-22763737754686614722007-12-14T16:32:00.000-05:002007-12-14T16:53:38.981-05:00aspell for emacs and Mac OS X<p>Here's how to add a spelling checker to Emacs on Mac OS X. This will work for both terminal based or GUI based versions.</p>
<p>First, for goodness sake, install <a href="http://www.macports.org/">MacPorts</a>. After it is installed from terminal do:
</p>
<pre>
$ sudo port -v selfupdate
$ sudo port install aspell-dict-en
</pre>
<p> You can install other languages too. Take a look at "<code>port search aspell</code>" for the list.</p>
<p> Then just add the following line to your <code>.emacs</code></p>
<pre>
(setq ispell-program-name "/opt/local/bin/aspell")
</pre>
<p>Load you your text (or html or xm) file and do <code>M-x ispell</code></p>
<p>Oh yeah! You can read more on the Emacs commands <a href="http://www.gnu.org/software/emacs/manual/html_node/emacs/Spelling.html#Spelling">here</a></p>nickghttp://www.blogger.com/profile/04659163622120256748noreply@blogger.comtag:blogger.com,1999:blog-8885167082319491081.post-32380539131636657672007-12-11T16:58:00.000-05:002007-12-11T17:08:07.276-05:00Tamperproof URLs and PHP<p>
I finally posted my slides for "Tamperproof URLS and PHP" from the NYPHP talk on 27-Nov-2007 at <a href="http://modp.com/slides/securestring/">http://modp.com/slides/securestring/</a>. In addition the source code and unit test are posted. This is a temporary spot until code.google.com bumps up my resources and then it will go into SVN.</p>
<p>Thank you all for coming. Based on the questions and feedback, I have completely revamped the slides and added a lot more examples. Enjoy! --nickg
</p>nickghttp://www.blogger.com/profile/04659163622120256748noreply@blogger.com