The writing continues on my own domain

I haven't blogged here for a while, but for the last few months I've been writing a blog on my own domain, https://erik.itland.no

You if you enjoyed my writing here you are welcome to continue reading at my new blog.

Things that will be lost on Windows reset (unless you backup)

Checklist:

• GPG4WIN 
• settings
• keys
• PuTTY 
• settings
• keys should be fine as long as they are stored in a ordinary folder

The other perspective

"If you look forward to go to work in the morning that counts for something as well, doesn't it?" - Succesful friend of mine answering questions from economics 101 some years ago.

What is going on with the search results?

When I see googlers in online forums defending the search results they often ask for data sets showing the problem. This isn't a data set but here should be some reproducible test cases that fails horribly for what was once the undisputed place to go for relevant search results.

Example 1: "sublime text 2" "focus group"

Query: "sublime text 2" "focus group"
Expected result: Documents including the word groups "sublime text 2" and "focus group".
First result: http://www.sublimetext.com/2
Problem: Irrelevant
Possible explantion: Extensive linking using  "focus group"as link text? (Doesn't prevent it from being utterly irrelavant.)

Example 2: cisco "anyclient"

Query: cisco "anyclient"
Expected results: A list of documents containing something like cisco + the exact word "anyclient"
First page of results: All results are for "AnyConnect". Which seems to be the correct product name but was not what I was searching for. I was searching to see if anyclient was a common mistake, if there had been a product called anyclient etc so I would not embarrass myself.
Problem: Silent rewrite of my query. I'm all for usability features like suggest, "Did you mean" etc. "Showing results for <something else>, did you mean <what you searched for>?" is a pita to me but I can see it being useful for less technically inclined people and people who hasn't developed the same kind of tunnel vision as me. Silently rewriting counts as bug or arrogance in my book.

Trying to get around these issues

Report the problem

This is obviously a bug so lets report it. Done. Months ago. No change yet.

Image showing Googles error reporting interface. Trying to report the problem is an exercise in futility. They may or may not fix it but on one point they are very clear:  They will never inform you. The only way you will ever know is if you see a change sometime.

Activate the "verbatim" search tool

 Turns out Google has thought of this. Here is the verbatim ("ordrett") search tool. Good idea, one minor flaw: It doesn't work!

No, "Backup Exec" is not "Shopify". And this is hardly what I'd call verbatim.

Using another search engine

I love competition. So I tried with a couple of other search engines.

DuckDuckGo also insists that BackupExec is relevant when searching for Shopify.

Recovering files from an unsaved project in Visual Studio

Woke up early this morning and decided to train a little more .Net. Only to find that the project was not available under recent files. In hindsight this is how I think it happened:

• I worked on an unsaved project without noticing it was unsaved.
• The machine lost power
• Windows didn't hibernate properly because the disk was crammed

 Luckily the files where available under Visual Studio 2010\Backup Files and getting my work back was as simple as creating a new project with a new Form with the same name that the old one had and then copy the old stuff into the new files.

So far I like Visual Studio as long as using Java feels equally wrong, why not? (loved to read Groklaws summary of yesterdays filings btw. If it ends the way it is heading now there might even be hope for the Java language.)

Review: Zenoss 4/5

Have been runnning Zenoss for over a full year, here is my verdict:

Product (4/5):

• Monitors hundreds of nodes running of a not to beefy vm.
• Easily extendable.
• Smooth upgrades. (Not automatic but well documented.)
• User interface is sometimes a little hard to understand. Not bad for a product the size of Zenoss but not perfect either.

Documentation (4/5):

• Lots of official documentation freely available covering most of what one needs to know.
• Forums were migrated to Jive a couple of years ago leaving lots of broken links.

Community (5/5):

• Nice and helpful community.
• Lots of contributed plugins a.k.a. ZenPacks. Examples 
• Hardware health status on HP servers
• Oracle databases
• vmware ESX and ESXi

Conclusion:

I did a lot of reading up front before implementing Zenoss and considered 

• Zabbix,
• Hyperic,
• OpenNMS, 
• Nagios 

 before making the decision. I personally liked both Zabbix and OpenNMS and have heard good things about the but Nagios/Icinga and Hyperic but chose Zenoss because of 

• agent-less architecture 
• hp hardware monitoring almost out-of-the-box
• snmp trap handling.

I recently asked myself  if I'd use Zenoss again and the answer is: Yes, at least if I have to monitor hardware.

However keep in mind that a full implementation might easily take weeks even for a skilled sysadmin, depending on the size of the monitored network, the status of the network (is snmp enabled? Who has to approve it? etc.)

Review: Flexiglass 5/5

If you are a Mac user that have spent any time on recent desktop Linux, chances are you will be missing Compiz-like window management.

(On a side note: Even Windows converts may be missing some niceties such as double click to maximize, or resize by any corner or window edge that wasn't added until Lion.)

I bought a Flexiglass license a couple of months ago and I love it. Flexiglass offers:

• Move a window using alt + two fingers (think alt-dragging in gnome/kde)
• Resize using alt + three fingers (Mac OS X > 10.7 can't resize a window by dragging any corner/border) 
• Real maximize by right clicking the plus button (Make a window fill the screen, instead of leaving to the window manager to guess how much space it needs. )
• Real close by two finger clicking the cross button. (Actually closes the application instead of closing the window and leaving the application running.)
• Shortcuts for resize/move to specified postitions. Nice for when you want a howto and a terminal side by side.

I have a Divvy-license from before but after I started using Flexiglass I don't need it anymore as Flexiglass offers all I used Divvy for and more.

Below are some screenshots from the settings to show the configurability of Flexiglass:

Basic settings for Move and Resize. The settings are customizable for both mouse and trackpad and Flexiglass automatiaclly changes profile when you attach or remove a mouse. Nice!

The exceptions list: I don't need it but I figure graphical artists might need to be able to use the mouse together with modifier keys in some applications without activating Flexiglass.

I thought I'd use the "Real Zoom" and the "Real Close" features a lot, but as it turns out I end up double clicking in the window title bar to maximize it instead. On the other hand I use the Quick Layout Shortcuts a lot for example to tile up two shells at the right of the screen and the documentation on the left

Autostart? Yes!

Do those stylesheets really need to be in the page body?

I once worked on a medium size enterprise software project that felt very sluggish and slow. I was new to the technology stack, new to anything but small web applications and I figured it just wasn't possible to get that particular stack any faster.

As the project went on features were added, more optimizations were added by expensive consultants,  a few performance regressions were weeded out but the app was still as sluggish as ever until one day the ux designer came and asked if there was any reason the consultants needed the stylesheet in the page body instead of in the head.

There wasn't, and at the next deploy the app was suddenly a whole lot faster and smoother. So much for the cruft of StringBuilder for every string concatenation. So much for higly efficient log statements, each spanning three lines, applied liberally across the whole codebase.

From that day on I became a little more pragmatic, a little less uninformed regarding that particular stack, the expensive consultants that happend to know it and optimizations for web applications.

Disclaimer: Of course, I'm not saying you should ditch StringBuilder and stop checking if debugIsEnabled() in thight loops, but in the average controller code in the average enterprise web project I guess they won't matter much.

Review: repositoryhosting.com 5/5

I've been using repositoryhosting.com since late august last year, and it seems like now is a about time to write a very small review.

The good stuff:

• Your choice of version control system:
• Git
• Mercurial
• Subversion
• Filestore with WebDAV included
• Trac
  Included plugins, selectable on a project by project basis:
• Agilo
• Timing and Estimation
• Discussion Forum
• Batch Modify Tickets
• Custom Roadmap
• Download Releases
• Markdown Macro
• Spam Blocker
• Table of Contents Macro
• Ticket Importer
• Wiki Backlinks Macro
• Workflow Editor
• Wysiwyg Editor
• Support for incoming mail
• Easy backups, both 
• hosted and
• easily downloadable (means no lock in)
• User-friendly frontend lets you easily
• create new repos, 
• add users, 
• administer permissions
• everything else you need to do
• Unlimited users
• Unlimited projects
• The price
• 2GB for 6$/month
• 1$/month/additional GBs

I use it for a couple of my spare time projects and I also have a kind of GTD task list in there.

The option to create tickets by mail is also really great. I guess almost every busy person has a couple of use cases for a working automatic mail-to-task-interface.

If you have read everything  carefully so far, you'll see that there is no list of bad stuff. The reason is I haven't found anything yet. (The closest thing I can come up with is if they will manage to stay in business at this price, when everyone else needs to charge more.)  I've only used it for 6 months though, so I might have missed something.

RewriteBase takes one argument

Staying in Denver, CO, away from my family, and far from the development databases at work I decided I had time to work on a pet project of mine.

Knowing there will be a few hours on airports and in the plane on my way home I decided to get the project running correctly on my laptop before I continued work.

Then I ran into this:

Internal Server Error

The server encountered an internal error or misconfiguration and was unable to complete your request.

Please contact the server administrator, you@example.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.

More information about this error may be available in the server error log.

Log file:

[Sun Jan 09 14:16:56 2011] [alert] [client ::1] /Users/Sites/web/.htaccess: RewriteBase takes one argument, the base URL of the per-directory context

After browsing through the first two pages of google results withouth finding anything useful I suddenly realized maybe the config files doesn't accept comments at the end of the lines. Turns out to be true. See http://httpd.apache.org/docs/1.3/configuring.html#syntax

Configuring Mac Snow Leopard for Symfony development

If you want to develop symfony applications using the built in Apache web server in Mac OS X here are my notes. Keep in mind, this is not a production configuration but just the necessary steps to get a developer workstation up and running.

1. Activate Web Sharing
   System Preferences -> Sharing -> Web Sharing
2. Change your UserDir settings. In a terminal window, edit the file with your userdir configuration.
   vim /etc/apache2/users/username.conf
   Remove or comment the contents. Paste the following information:
   UserDir Sites/web
   
   
   <Directory "/Users/username/Sites/web">
       Options Indexes MultiViews
       AllowOverride All AuthConfig
       Order allow,deny
       Allow from all
   </Directory>
   
   Alias /~username/sf /Users/username/Sites/lib/vendor/symfony/data/web/sf
   
   <Directory "/Users/username/Sites/lib/vendor/symfony/data/web/sf">
     AllowOverride All
     Allow from All
   </Directory>
3. Substitute the text username for you username.
4. Enable php (and mysql if you want). See http://stackoverflow.com/questions/1293484/easiest-way-to-activate-php-and-mysql-on-mac-os-10-6-snow-leopard
5. Now you can either install
   
   • the sandbox
   • the svn version,
     
   • the tar.gz version
   • or you can check out your existing project from source control.
6. After you've installed symfony, edit the .htaccess file in the symfony web folder
   
   • Enable RewriteBase by uncommenting it
   • Change the RewriteBase directive  to RewriteBase /~username/
7. Restart apache
   sudo apachectl restart
   

BTW: I wrote a more detailed post back in 2008. It hasn't been updated since then, but there might still be a few useful hints there: http://techinorg.blogspot.com/2008/07/ubuntu-post-install-customization.html

The web2py ide

The web2py ide is Firefox. Or Chrome, or Opera, or Safari (or Internet Explorer).

Just create a good old frameset and you can have split view between models and controller, documentation and even a preview running in one browser tab. Running the browser in full screen and the built in code editors in "full screen" (which fills the whole assigned frame), this turned out to work really well.

Example frameset:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"
"http://www.w3.org/TR/html4/strict.dtd">
<html lang="en">
<head>
<title>web2py ide - 0.1 beta :)</title>
</head>
<FRAMESET ROWS="75%, *" COLS="*, 40%">
<FRAME SRC="http://127.0.0.1:8000/admin/default/peek/yourapp/models/db.py">
<FRAME SRC="http://127.0.0.1:8000/admin/default/design/yourapp">
<FRAME SRC="http://127.0.0.1:8000/yourapp/default/index">
<FRAME SRC="http://127.0.0.1:8000/yourapp/appadmin/index">
</FRAMESET>
</html>

Can you see some quick improvements? Post them in the comments!

Quick hack to develop web2py on a different machine

For security reasons web2py will not allow you to log in as administrator unless you are logging in from the same workstation. (There are exceptions, but they are cumbersome: See the official book for full details.)

However, there is a quick hack that allows you develop from any machine: Just set up a ssh tunnel:

ssh username@ip_address_or_dns_name -L 8000:127.0.0.1:8000 -N

As usually, I didn't come up with this myself. This time I found it here: http://www.revsys.com/writings/quicktips/ssh-tunnel.html

(However, if you read a few of the samples available on the internet, you'll see that may of them miss the point that in the local_port:ip_address:remote_port part you can and should use 127.0.0.1 instead of the external ip address. The reason is that this is the address that the request seems to come from in the server program at the remote computer.)

Using web2py, first impressions

I've had the opportunity to do some quick prototyping in web2py in my spare time over the last few weeks. Here are some very quick notes about what to like and what not to like:

Good:

• Works out of the box. Just download, open, choose password, press start. Your default browser starts automatically.
• Automatic updates (beta warning, bur works.)
• Built in editor and version control.
• Built in user registration and management.
• Automagic migrations (connecting to legacy databases are supported and documented, too)
• Built in support for multiple databases, including distributed transactions.
• Built in sqlite database, and postgres, mysql and even mssql seems to be just one config setting away.
• The whole official book is available online.
• Author intends to keep the api stable. (For anybody who has had the joy of supporting an application over multiple years and multiple versions of the same framework.)
• Bonus point 1: Nice default look and feel, and it is easily customizable. It's just html with placeholders.
•  Bonus point 2: Applications built on top of web2py can be packed, downloaded and installed on another instance.

Needs improvement:

• The documentation. There is lots of official documentation, but the search engine is next to useless, and google picks up old, outdated blog post before the official documentation.
•  Routing: From the official documetation: The url http://127.0.0.1:8000/examples/default/status/x/y/z?p=1&q=2 will presented to the controller as:
• request.args ['x', 'y', 'z']
• request.get_vars <Storage {'p': 1, 'q': 2}>
  Why you'd like to separate between those are beyond my understanding. I always thought unifying those was one of the features of modern web frameworks.

Preliminary conclusion:


web2py is production ready and is already used a few public facing sites. To me it seems like a perfect match for small "intranet applications" but it is in no way limited to this.

Although I've not programmed apex myself, it often comes to mind because of features like online editing and redistributable applications built on top of it.

Getting ipythonx to work with Mac OS X

If you're trying to get ipythonx to work under Mac OS X 10.6 Snow Leopard you might get and error message like this:

ImportError: /System/Library/Frameworks/Python.framework/

Versions/2.6/Extras/lib/python/wx-2.8-mac-unicode/wx/

_core_.so: no appropriate 64-bit architecture (see "man 

python" for running in 32-bit mode)

Here's a couple of hints:

1. Use standard ipython instead. Unless there was something I didn't catch, ipython seems to better than ipythonx.
2. If you want to try anyway here's the easiest way I found: In a terminal session:

   1. export VERSIONER_PYTHON_PREFER_32_BIT=yes

   2. python `which ipythonx`

Copy OpenVPN settings from Tunnelblick on Mac OS X to Windows 7

I forgot the charger for my job laptop in the office today. So I spent the last few minutes of battery copying the configuration files to the pc at home.

Below is what I ended up doing:

1. Install OpenVPN software. You can find it on the official OpenVPN site.
2. Copy all the config files from the job laptop to C:\Program Files (x86)\OpenVPN\config. The job laptop uses tunnelblick, so the files was located under
   ~/Library/Application Support/Tunnelblick/Configurations
3. Added .opvpn to the file name for the main configuration file (the one that is not .crt, .csr or .key)
4. Change to Windows style line endings. I did this by opening the files in WordPad, copying the contents to a Notepad window and saving.
5. Starting the client software.

Of course: There was a lot of trial and error. I guess I ended up using 30 minutes tinkering and 30 minutes blogging.

Gotchas:

• Make sure all line endings are corrrect. You can see this easily by opening the files in Notepad.
• Make sure you run the OpenVPN program as administrator.

Swithing to Mac - one year after

This has been a long time coming, but here it is: My "Switching to Mac story".
I've been working on this for almost a full year. It started with three optimistic blog posts about the things I learned while unlearning Windows, and then reality struck me: I thought Mac supposed to be almost like Linux, supported. It is not.
Here is a nice summary of the differences between Mac and PC: Top Ten Things They Don’t Tell You About Switching. Below are just my additions.

The keyboard:

+

• it is backlit

-

• weird layout (ctrl/fn keys are switched and there is no setting anywhere to get them back)
• missing keys (page up/down, home/end, print screen) comes to mind.
• I can hear what some of you are thinking: It's just a matter of taste. Well, a normal (not laptop) Apple keyboard has ctrl in it's correct position, so it's obviously not a usabilty thing or a design thing but rather a "we don't care"-thing.

Software:

+

• many small companies offer all kinds of small programs for mac

-

• every program looks different
• few of them is anywhere near usable without a mouse or touchpad. Placing all menus on top of the screen makes this even worse.
• often you'll find functions hidden away behind a gear icon that may hide anything. No visual clue until you click on it. Same goes for the strange button at the top rigt of every window that will hide something.
• Don't expect free and open source software that runs on both Windows and Linux to be available on Mac. Some programs, like Gimp, runs somehow, but looks really ugly as they are running in some kind of X11 terminal.
• Even on programs that are available, you can expect to wait a little longer before new releases are available, and you'll also have to expect a few extra bugs. Of course this is not Apples fault, but you might like to keep it in mind anyway as you might be going to suffer from it.

Will I ever buy another Mac? Not likely. Not until they fix the keyboards and keybindings.
A few things comes to mind as for why I'd prefer a Mac over a standard pc running Linux, though:

• iTunes (best way to buy music in Norway, where I live)
• MagSafe power
• Good screen
• Less likely to break down while at a customer site

Appendix: The hopeless keyboard addicts guide to the Mac keyboard:

I was about to make a table here, detailing how to get at least home/end and pageup/down in the most used programs. But then some programs even differ inside of the program (like, "home" and "end" are mapped to different key combos depending on if you are located in the address bar of you browser or in a text area. This is especially funny when you've almost finished some writing on a blog, a company intranet etc, you want to select one line of text to move it around, and you forget that the key combo that means mark to start of line in one application means go to previous page in the browser. This particular one has hit me twice.)
Here are a few hints anyway:

• ctrl+a go to start of line. Works in almost every program, including terminal, but doesn't work together with shift.
• ctrl+e go to end of line. Works in almost every program, including terminal, but doesn't work together with shift.
• cmd + arrows/fn + arrows - may or may not works for either home/end. Depends on mood of programmer, moon phase when program was written or some other, unknown parameter. If one doesn't work, try another, they rarely do anything really wrong except triggering the web browser back action.
• cmd+shift+4 - screenshot, choose area. Press space to select a whole window.

The real reason why we don't use silverlight or flash

I think I speak on behalf of a lot of people if a say:

A lot of silverlight & flash fans are missing the point: We know about flash. It's not that we somehow do not "get" silverlight. We program Java, PHP, Python, -list goes on and for many of us also includes C# and actionscript. We do use a lot of different frameworks. Learning new stuff is part of our daily grind (and to be honest, we mostly love it.)

We also do struggle with fonts that might or might not be available, css rendered differently in IE etc. We just don't want to add dependencies to

• big,
• cpu-hogging,
• closed source plugins
• that might or might not be supported on our future favourite operating system

to our nicely crafted and cross browser compatible web sites. Hey, -we might not even be allowed by the customer to do that, and even if we were, I think most of us would explain the customer/boss in question why they should be careful to do that.

Simple recovery of deleted files from ext3 file systems

Yesterday I finished recovery of about 8000 photos by burning them to DVD and handing them over to a friend of mine.

Recovery of files from ext 3 partitions have long been considered somewhere between hard and impossible, but thanks to free and open source PhotoRec the recovery was done automatically during the night after answering a few, relatively simple questions. Next day I just rsynced the files to my laptop, splitted them into four folders about to fit into DVDs and burned them. It even read the files directly from the dumped img file.

The burning process was actually the one that took most time.

There's one pitfall I'd like to mention, though:

PhotoRec first does a scan to find the block size of the disk in question, and then reads the disk block by block, matching the beginning of each block with a signature database.

The first time I ran the program I chose to scan the entire drive. After scanning the entire drive it had found less than 50 photos. I decided to give it one more try, this time with just one partition. I chose the partition I considered most likely to contain the images, set the paranoid setting to false and chose photos (jpg) only.

This time it found more than 40 000 jpgs. Most of them seemed to be browser cache etc so I filtered out everything smaller than 1 MB.

Because I changed two parameters I can't say why this happened but my best guess is that it happened because the block size was different between the two partitions.

Spam and malware, with screenshots

Got this fake antivirus message yesterday and thought it might be worth sharing:
This one was rather well crafted, but they obviously didn't take the time to check operating system, even though they found my ip address. (See bottom left.) Here's another one. This one I got in mail a few weeks ago, from a friend who obviously got her email account stolen:

It's in norwegian, and it's readable, but it is quite obvious that she didn't write it. I guess they fed it through google translate. Translated back to english it would become something like (errors intended): "I need you help NÅ How is you? ..." Wonder what this is going to look like in a couple of years when automated translation gets getter : )