I’ve been using digital ocean for a couple of years for my own business and for some clients. A few months back I started a new personal account with a privacy focus to run my own VPN etc.

My credit card on file was maxed so I went to add a new card. After accepting my new card, I got a notice and my account was frozen: Automated Account Verification Request. Digital ocean requested the following information:

* Name
* Address
* Type of Traffic
* Blog/Wesite
* Phone Number
* Social network (twitter, facebook etc).

Really?, my facebook and blog? This seemed very intrusive to me. My credit card authorization is proof of who I am. When moving the post office uses a credit card to verify who you are. So if it is good enough for the US Government, why not digital ocean?

Although it gave me an icky feeling, I complied with their request. It seemed overly intrusive but it was late and I wanted to get back to real work.

To help us verify your identity, please send us a scan or photo of your government-issued photo ID or passport 

Really? In addition to my social networking sites you need my drivers license too? Digial Ocean is now trying to collect more information than my bank.

Who are these people? If they had simply asked for my drivers license to begin with I would have complied. But they feel they need my social networking sites too?

I would worry when your ISP wants more information the the federal government or your bank for “identity verification”.

I have now moved all my services to http://www.vultr.com/. Vultr is great, and one only needs some BTC and an email address to use their service.

Aloha,

When updating to the latest macports, openssl 1.0.1 is installed. Then whenever Ruby tries to connect to facebook, you get errors like:

Errno::ECONNRESET (Connection reset by peer – SSL_connect)

port installed | grep  openssl
  openssl @1.0.0h_0
  openssl @1.0.1_2
  openssl @1.0.1b_0 (active)

Ok Great, we have the old version:

sudo port activate openssl @1.0.0h_0

Oh, but what if you don’t have the correct version already installed? Thats a little trickier:

svn co -r  90715 https://trac.macports.org/browser/trunk/dports/devel/openssl
cd openssl
sudo port install

Update: Installing Openssl 1.0.1c and rebuilding ruby also fixes the problem

sudo port selfupdate
sudo port update outdated
sudo port uninstall ruby19
sudo port install ruby19 +doc+nosuffix

And now I have ruby 1.9 and openssl 1.0.1c talking to facebook without peer resets.

References:

A hui hou!

I’ve been meaning to attend a rails conference and after moving to Oahu I figured it would be a while. But now the Rails conference is coming to me! So I couldn’t pass and will be at Aloha on Rails.

I hope to see you there!

Announcing MongooseDaemon, an objective-c wrapper for the wonderful mongoose http server.

It can be indispensable to be able to explore your iPhone apps directory structure when developing and debugging iPhone applications. You may also want to serve up content from your iPhone application via http.

Erica Sadun’s cookbook has an example of a hand rolled http server. However, it is only an example and is incomplete. I recently needed a http server in an iPhone app and after playing with Erica’s example I quickly realized I didn’t want to be in the business of creating a complete http server.

So I started looking for an existing http server I could embed in my app. I quickly found moongoose.

With just a few minutes of coding, I was able to get mongoose working in my iPhone application. To make it even easier for the next developer, I extracted my wrapper into the MongooseDaemon class and am offering it under a BSD license.

Here is some example code taken directly from one of my apps I am debugging:

1
2
3
4
5
6
7
8
9

  #import "MongooseDaemon.h"
  ....
  @class MongooseDaemon

  @interface MyAppDelegate : NSObject <UIApplicationDelegate, UITabBarControllerDelegate> {
    ...
    MongooseDaemon    *mongooseDaemon;
    ...
  }

1
2
3
4
5
6
7
8
9
10
11
12
13
14

  ...
  @implementation LightyAppDelegate
  ...
  - (void)applicationDidFinishLaunching:(UIApplication *)application {
  mongooseDaemon = [[MongooseDaemon alloc] init];
  [mongooseDaemon startMongooseDaemon:@"8080"];

  ...
  - (void)dealloc {
  ...
    [mongooseDaemon stopMongooseDaemon];
    [mongooseDaemon release];
  ...
  }

Now you can surf to your application. For example, if your iPhones IP on wifi is 192.168.1.100 (sorry a helper method for determining your iPhones IP will be coming soon), then you could surf to http://192.168.1.100:8080/

Serve files in good health!

 [301] tako:~>  ratos

App Name:TheSnapper
App Path:/Users/face/proj/TheSnapper

Paste xcode stack trace to stdin, then type 'sim' or 'arm' on a line by itself.
  Type 'exit' or '^D' to quit and 'app' to print the current app.

ratos>2009-02-05 10:21:31.107 TheSnapper[751:20b] Stack: (
    808221155,
    806100816,
    808224973,
    807957033,
    807851552,
    25785,
    22599,
    22855,
    807923363,

arm

-[BackViewController doubleSnapAction:] (in TheSnapper) (BackViewController.m:192)
-[BackViewController resetAllSiders] (in TheSnapper) (BackViewController.m:126)
-[BackViewController resetSnap:] (in TheSnapper) (BackViewController.m:138)

When I first learned iPhone development it took me a while to find atos. Exception handling comes natural but Xcode spits out addresses in their stack traces. After much googling I finally found atos. I then immediately found Apple’s docs to integrate atos into an exception handler. Cool….NOT!

Turns out Apple’s Xcode integration relies on /usr/bin/atos at runtime….not possible on my iPhone.

So, I immediately wrote a quick and dirty shell in Ruby that takes a stack trace in from xcode, and prints out any lines that have symbols. Gobbledygook in, nice human readable stack trace out!

After enjoying ratos for the past few months I finally had a chance today to clean it up a little and publish it on github.

git clone git://github.com/face/ratos.git
sudo cp ratos/bin/ratos /usr/local/bin/.

# csh
setenv RATOS_APP_PATH ~/proj/TheSnapper

# or bash
export RATOS_APP_PATH=~/proj/TheSnapper

 [303] tako:~>  echo $RATOS_APP_PATH
/Users/face/proj/TheSnapper
 [304] tako:~>  ratos

App Name:TheSnapper
App Path:/Users/face/proj/TheSnapper

Paste xcode stack trace to stdin, then type 'sim' or 'arm' on a line by itself.
  Type 'exit' or '^D' to quit and 'app' to print the current app.

ratos>exit

 [305] tako:~>  env RATOS_APP_PATH=~/proj/IttyBooks ratos

App Name:TheSnapper
App Path:~/proj/IttyBooks

Paste xcode stack trace to stdin, then type 'sim' or 'arm' on a line by itself.
  Type 'exit' or '^D' to quit and 'app' to print the current app.

ratos>exit

 [306] tako:~>  env RATOS_APP_PATH=~/proj/IttyBooksFree RATOS_APP_NAME=IttyBooks ratos

App Name:IttyBooks
App Path:~/proj/IttyBooksFree

Paste xcode stack trace to stdin, then type 'sim' or 'arm' on a line by itself.
  Type 'exit' or '^D' to quit and 'app' to print the current app.

ratos>exit

 [307] tako:~>  ratos

App Name:TheSnapper
App Path:/Users/face/proj/TheSnapper

Paste xcode stack trace to stdin, then type 'sim' or 'arm' on a line by itself.
  Type 'exit' or '^D' to quit and 'app' to print the current app.

ratos>2009-02-05 10:21:31.107 TheSnapper[751:20b] Stack: (
    808221155,
    806100816,
    808224973,
    807957033,
    807851552,
    25785,
    22599,
    22855,
    807923363,
    816119156,
    816119004,
    816157144,
    8381,
    8244
)
terminate called after throwing an instance of 'NSException'

arm

-[BackViewController doubleSnapAction:] (in TheSnapper) (BackViewController.m:192)
-[BackViewController resetAllSiders] (in TheSnapper) (BackViewController.m:126)
-[BackViewController resetSnap:] (in TheSnapper) (BackViewController.m:138)
main (in TheSnapper) (main.m:14)
start (in TheSnapper) + 52

ratos>exit

Enjoy!

I used objective-c activerecord in IttyBooks as my ORM layer.

I fixed some bugs and made some API changes. Most of the API changes where for performance or memory reasons. While activerecord is not as feature complete as its Ruby cousin, it was nice to contribute. Sure beats mixing SQL in objective-c as well.

Anyway, you can view all of my changes on my fork at github. And they should end up merged back in aptiva’s master tree as well.

Gota love github!

Please check them out!

We have extracted ActivityStreams from Legaltorrents.com and released it under the BSD license.

This is a new plugin for the Rails Community that provides a customizable framework for cataloging and publishing user activity and social objects. We currently aim to provide support for microformats in HTML, Atom feeds, and compatibility with the open source DISO social networking implementation for activity discovery and consumption.

Here is a quickie for Rails 2.1:

./script/plugin install git://github.com/face/activity_streams.git

It may work on earlier versions of Rails as well. Don’t have git:

./script/plugin install svn://rubyforge.org/var/svn/activitystreams/trunk

For further documentation here is the rdoc and the AcitivityStreams Home Page, or just show me the code.

development:
  adapter: mysql
  database: legaltorrents_development
  username: fred
  password: password
  socket: /tmp/mysql.sock

production:
  adapter: postgresql
  database: legaltorrents_production
  username: fred
  password: password
  host: localhost

Thanks to Ruby on Rails, transferring and converting database from one database platform to another only takes a few lines of code. There are rake tasks for dumping to YAML and back. However the existing YAML scripts I found had issues with some of our data and then failed for blobs. This script will only work with a “rails style” database. By “rails style” I mean any database where every table has a unique key named “id”.

A special thanks to Matson Systems, Inc. for having me write this script for LegalTorrents and then contribute it to all under a BSD license.

1
2

rake db:migrate
env RAILS_ENV=production rake db:migrate

1
2
3
4
5

cd lib/tasks
wget 'http://github.com/face/rails_db_convert_using_adapters/tree/master%2Fconvert.rake?raw=true' -O convert.rake

# and run it
rake db:convert:prod2dev

Update Oct 1, 2008:Fixed a bug today for Rails 2.1.1. Also got rid of the hash of data that was a relic from an early version of the script that used a single model object.

Here is the full code to convert.rake:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114

#
# Convert/transfer data from production => development.    This facilitates
# a conversion one database adapter type to another (say postgres -> mysql )
#
# WARNING 1: this script deletes all development data and replaces it with
#                     production data
#
# WARNING 2: This script assumes it is the only user updating either database.
#                     Database integrity could be corrupted if other users where 
#                     writing to the databases.
#
# Usage:  rake db:convert:prod2dev
#
# It assumes the development database has a schema identical to the production 
# database, but will delete any data before importing the production data
#
# A couple of the outer loops evolved from 
#    http://snippets.dzone.com/posts/show/3393
#
# For further instructions see 
#    http://myutil.com/2008/8/31/rake-task-transfer-rails-database-mysql-to-postgres
#
# The master repository for this script is at github:
#    http://github.com/face/rails_db_convert_using_adapters/tree/master
#
#
# Author: Rama McIntosh
#         Matson Systems, Inc.
#         http://www.matsonsystems.com
#
# This rake task is released under this BSD license:
#
# Copyright (c) 2008, Matson Systems, Inc. All rights reserved.
# 
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
# 
# * Redistributions of source code must retain the above copyright
#   notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
#   notice, this list of conditions and the following disclaimer in the
#   documentation and/or other materials provided with the distribution.
# * Neither the name of Matson Systems, Inc. nor the names of its
#   contributors may be used to endorse or promote products derived
#   from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
# FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
# COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
# BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
# ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.

# PAGE_SIZE is the number of rows updated in a single transaction.
# This facilitates tables where the number of rows exceeds the systems
# memory
PAGE_SIZE=10000

namespace :db do
  namespace :convert do    
    desc 'Convert/import production data to development.   DANGER Deletes all data in the development database.   Assumes both schemas are already migrated.'
    task :prod2dev => :environment do

      # We need unique classes so ActiveRecord can hash different connections
      # We do not want to use the real Model classes because any business
      # rules will likely get in the way of a database transfer
      class ProductionModelClass < ActiveRecord::Base
      end
      class DevelopmentModelClass < ActiveRecord::Base
      end

      skip_tables = ["schema_info", "schema_migrations"]
      ActiveRecord::Base.establish_connection(:production)
      (ActiveRecord::Base.connection.tables - skip_tables).each do |table_name|

        ProductionModelClass.set_table_name(table_name)
        DevelopmentModelClass.set_table_name(table_name)
        DevelopmentModelClass.establish_connection(:development)
        DevelopmentModelClass.reset_column_information
        ProductionModelClass.reset_column_information
        DevelopmentModelClass.record_timestamps = false

        # Page through the data in case the table is too large to fit in RAM
        offset = count = 0;
        print "Converting #{table_name}..."; STDOUT.flush
        # First, delete any old dev data
        DevelopmentModelClass.delete_all
        while ((models = ProductionModelClass.find(:all, 
            :offset=>offset, :limit=>PAGE_SIZE)).size > 0)

          count += models.size
          offset += PAGE_SIZE

          # Now, write out the prod data to the dev db
          DevelopmentModelClass.transaction do
            models.each do |model|
              new_model = DevelopmentModelClass.new(model.attributes)
              new_model.id = model.id
              new_model.save(false)
            end
          end
        end
        print "#{count} records converted\n"
      end
    end
  end
end

As some of you know I’ve been working full time for Matson Systems, Inc. building out LegalTorrents. I must apologize I have been neglecting parts of my blog. Fortunately, I’ve been swamped building out cool features for LegalTorrents and Matson wants to contribute back. After this caching article look for a rake task to convert a Rails app from one database platform to another, then a plugin for generating Activity Streams.

As a new community, we have to support tens of thousands of registered users. Yet on any given news day we need to support hundreds of thousands of non registered users.

Let’s begin with conditional caching in Rails 2.1 (if 2.0, see below). Conditional action caching is a new feature of the Rails 2.1 API. First off, pre Rails 2.1 the default was disk. In rails 2.1, the default is RAM. Not going to work on limited resources:

1
2
3

# Put this in RAILS_ROOT/config/initializers/something.rb
ActionController::Base.cache_store = :file_store, "#{RAILS_ROOT}/tmp/cache"

1
2
3
4
5
6
7
8

class ApplicationController < ActionController::Base
  # ...
  protected
  # of course logged_in? needs to be defined...restful_authentication is what I recommend.
  def do_caching?
    !logged_in? && flash.empty?
  end

1
2
3
4
5
6

class TorrentsController < ApplicationController
  # ...
  caches_action :show , :if => Proc.new { |controller|
                          controller.send(:do_caching?) }

1
2
3

# This cron entry that runs every 10 minutes and removes any files older than 10 minutes named '*.cache'
3,13,23,33,43,53 * * * * find /home/ltdeploy/legaltorrents/tmp/cache -mmin +10 -name '*.cache' -exec rm -f {} \;

1
2
3
4
5
6
7
8
9
10

class CategoriesController < ApplicationController

  caches_action :show, 
    :if => Proc.new { |controller| controller.send(:do_caching?) }, 
    :cache_path => Proc.new { |c|
      c.params[:page] ?
      "#{c.request.host}.#{c.request.port}/#{c.send(:category_path,c.params[:id])}/page/#{c.params[:page]}" :
      "#{c.request.host}.#{c.request.port}/#{c.send(:category_path,c.params[:id])}/page/1"
    }

End of Story for Rails 2.1

Now, Rails 2.0 doesn’t have :if in caches_action. To work around this we used a simple monkey patch:

1
2
3
4
5
6
7
8

class ApplicationController < ActionController::Base
  # ...
  protected
  # Overrides Rails core to do action_cache when not logged in...Only works in Rails 2.0 and maybe earlier
  def perform_caching
    @@perform_caching && !logged_in? && flash.empty?
   end

1
2
3
4

class TorrentsController < ApplicationController
  # ...
  caches_action :show

I was so excited, Apple and AT&T approved tethering for my iPhone…but it was gone, no wait, gizmodo say’s it’s back! (click on picture to the left which is the “App Store” link in the gizmodo article to install NetShare right now via iTunes). Update:The link is broken now, NetShare is not available in the US app store. For us lucky few who purchased it….it still works…

However, after installing NetShare via iTunes onto my iphone I could not get it to work. I followed the instructions only to get:

Safari can’t open the page. Safari can’t open the page “http://www.apple.com/”. The error was: “unknown error” (NSURLErrorDomain:-1) Please choose Report Bugs to Apple from the Safari menu, note the error number, and describe what you did before you saw this message.

After fiddling with it for a few minutes I found a solution. The iphone appeard to be still trying to using the WiFi connection and not 3G.

So, right before the final step of launching NetShare on the iphone (i.e. after setting up the adHoc network on the iphone to your PC), launch Safari on the iPhone and surf to a page. This will cause the little WiFi icon to change to 3G at the top of your phone. Now lanuch NetShare that you have the 3G icon back and everything will work like a dream.

I can even use the ssh client on my OSX laptop to connect to remote servers using connect.c to proxy ssh through the iphone. To get this to work was something like:

1
2
3
4
5
6

wget http://www.taiyo.co.jp/~gotoh/ssh/connect.c
gcc connect.c -o connect -lresolv
sudo cp connect /usr/local/bin/.
ssh -o 'ProxyCommand /usr/local/bin/connect -5 -S 10.10.10.1:1080 %h %p' 206.71.190.75
# Of course the ProxyCommand can go in your ~/.ssh/config

Have iPhone, Have Laptop, will travel!

OT – I’ve been doing most development on Ubuntu and OpenBSD. I recently made the mistake of ordering a fully loaded Dell Inspiron 1720, only to learn that Dell dumbed it down by replacing the DVI port with VGA and disabling the RAID support on the mother board. Instead of following Dell’s marketing plan and getting their XPS, I promptly went to Apple.com and bought a Macbook pro. I recommend development on the Macbook pro and Leopard is a great Unix development environment.

Okay, back on topic….My favorite solution to adding back the bookmark icons was a comment by klugerama in the comments of Lifehacker’s article Mac Tip: Add favicons to the Firefox bookmark toolbar

1
2
3
4
5

/* Kill(display: none) or show (display: inline) bookmark icons in the Personal Toolbar */ 
  toolbarbutton.bookmark-item > .toolbarbutton-icon {
  display: inline !important;
}

cd ~/Library/Application\ Support/Firefox/Profiles/c25zk8xx.default/chrome/
cp userChrome-example.css userChrome.css

1
2
3
4
5
6

/* change space around bookmark toolbar icons */
  #personal-bookmarks toolbarbutton {
    margin-left: -3px !important;
    margin-right: -3px !important;
}

1
2
3
4
5

/* Hide the Folders in the Personal Toolbar */
  toolbarbutton.bookmark-item[container] > .toolbarbutton-icon {
  display: none !important;
} 

I have a feeling this will help some of y’all if you are getting the following error:

Sorry! Something is not quite right with the request we received from the website you are trying to use. Please try again in a few minutes. If this error persists, please contact the site administrator for the website you are trying to use. If you are the site administrator, click here to contact us.

I get this error if I try to login on my development environment because localhost:3000 just won’t cut it for Yahoo’s OpenID security policy. If I run from a production URL on port 80, say http://myutil.com/ then signin works (though I haven’t gotten Simple Registration Attribute Exchange working with Yahoo).

Yahoo! Security Policies Yahoo! will only support Relying Parties running on webservers with real hostnames (IP addresses are not supported) running on standard ports (Port 80 for HTTP and Port 443 for HTTPS).

Hope this saves ya some time!

Update July 16th, 2008: Sphinx sphinx-0.9.8 has been released and this port has been updated.

Here is a OpenBSD port of Sphinx, Free open-source SQL full-text search engine. Sphinx is a wonderfully fast and memory efficient deep text search engine. I have found integrates nicely with Ruby and Ruby-On-Rails.

I hope at some point to get this port committed to the OpenBSD CVS repository. In the meantime this will be the ports home. Till it hits the OpenBSDs repository, it will live in my SVN repository here.

You can download a snapshot of the source code for the port here: sphinx.tgz. So far this port has been tested with MySQL and Postgresql on OpenBSD 4.2 and OpenBSD 4.3-betaOpenBSD-4.4-beta on the i386 platform with the Ultrasphinx Ruby-On-Rails plugin.

Instructions to build the port

If your ports tree is not already prepped, please begin by prepping your ports tree.

1
2
3
4
5
6

sudo su
export PKG_PATH="ftp://ftp2.usa.openbsd.org/pub/OpenBSD/4.3/packages/i386"
# or, setenv PKG_PATH "ftp://ftp2.usa.openbsd.org/pub/OpenBSD/4.3/packages/i386"
pkg_add libiconv mysql-server mysql-client
exit

Now we are ready to build the sphinx port:

1
2
3
4
5
6

cd /usr/ports/textproc
lynx --source "http://myutil.com/ports/4.4/sphinx.tgz" > sphinx.tgz
tar xvfz sphinx.tgz 
cd sphinx
make install  # or make package

1
2

env FLAVOR=pgsql make install

Thanks, and feedback is always welcome!

1
2
3
4
5
6
7
8
9
10
11

--- extconf.rb.orig     Tue Feb 26 12:03:53 2008
+++ extconf.rb  Tue Feb 26 11:52:16 2008
@@ -1,5 +1,7 @@
 require "mkmf"
 
+$LIBS += " -lstdc++ -laspell"
+
 have_header("ruby.h")
 have_header("aspell.h")
 have_library("aspell")

I recently installed the raspell gem on my development box as part of an evaluation of ultrasphinx.

Unfortunately, the native extension to raspell doesn’t link in the dynamic libraries it uses. A fix is provided below.

Note, using the raspell with ultrasphinx also caused a ruby core dump, this time in an assertion from the configuration for aspell. OpenBSD uses aspell-0.50.5 and after upgrading aspell to 0.60.5, raspell started working fine.

** Starting Rails with development environment...
ruby:/usr/local/lib/ruby/gems/1.8/gems/raspell-1.1/lib/raspell.so: undefined symbol 'new_aspell_config'
lazy binding failed!
/home/face/urevz/vendor/plugins/ultrasphinx/lib/ultrasphinx/spell.rb:33: [BUG] Segmentation fault
ruby 1.8.6 (2007-09-24) [i386-openbsd4.2]

A little poking around and I noticed the native library, raspell.so, doesn’t link in libaspell!

A quick fix is to add the libs to extconf.rb and rebuild the library:

1. cd /usr/local/lib/ruby/gems/1.8/gems/raspell-1.1/ext
2. edit extconf.rb and add the line $LIBS = ” -lstdc+ -laspell”
3. sudo make clean
4. sudo make
5. sudo cp raspell.so ../lib/.