Solr

Sunspot is the up and coming solution for integrating Solr into Ruby on Rails, and fortunately enough, the 1.0 release (followed quickly by 1.0.1!) has just come out last week.  Between acts_as_solr and Sunspot, Sunspot wins hands down for it’s support of a master/slave Solr configurations, embedded Solr for testing, richer indexing semantics, and not being tied to ActiveRecord.  The companion sunspot_rails gem does give wonderful ActiveRecord integration however.

Solr cores are the bees knees!  We’ve built a simple RoR webapp using HTTParty and the Solr API that allows you to perform all the admin functions for cores, and allows you to quickly clone a core for your own nefarious purposes!  Simplifies hacking around with a new schema or configuration without having a local copy of Solr running.  Allows multiple QA environments to potentially share a single Solr infrastructure.

Solr master and slave setup in a single VM.  While pointless from a scaling perspective, it’s a really great way to work out the kinks!  It’s funny to see a slave core polling the same Solr VM its in for updated segments!

JRuby

Doesn’t suck after all.  Actually, maybe I should say that JBoss, when combined with JRuby, means that JBoss doesn’t suck so much.  I had the aforementioned Solr core admin tool bundled up as a WAR file with JRuby, and was able to deploy it to an existing environment that had JBoss installed!  I didn’t have to install ruby on the box, (or JRuby for that matter!)  I just deployed the WAR file and bamn, off to the races.  Ops folks get the JBoss they love, I get the Ruby on Rails that I love.

And on a related note, Warbler was the key to thinking JRuby is cool.  I’d never actually had to package up a RoR app, so Warbler came to the rescue.  And you know what?  It was nice to build a single file that I knew had everything that I needed in it that could be scp’ed around!  And thanks to some cool code in the environment.rb, my app was able to load up the right configuration file for the environment based on an environmental variable set in JBoss.

Virtual Machines

I recently migrated a Linux VPS based RoR + Solr app (see a trend in tech choices )  to a Windows environment.  And to deliever the new Windows environment, I used VirtualBox to host the Windows Vista environment on my Mac laptop.

A couple of notes:

• VirtualBox may not have all the snazzy integration points of Parallels with the host computer like seamless application sharing, but it seems to be much lighter weight.  Starts up quicker, and I don’t get the spinning beach ball of death as much.
• If you are shipping a 11 GB file, you can’t use a 16 GB USB Memory Stick…  Turns out the biggest file is 4 GB.  (Although I never tried formatting the stick as NTFS, maybe that would have allowed a single 11 GB file???)
• Uploading 11 GB to a remote out on the internet server will take a long long long time.  Even on a really fast network. connection.
• If you need to format an external USB hard drive as NTFS on a Mac, it is possible!  Just fire up your trusty Windows Vista image in Parallels, plug the USB drive in, download and install the correct USB drivers so the drive doesn’t show up as a network share mapped to the Mac, and then use the built in reformatting tools!  Warning: This will take a loooong time!
• Lastly, if you are using VirtualBox, and you attempt to create a Windows XP machine, and attach a Windows Vista hard disk image to it, VirtualBox will let you!  And then Windows won’t start.  sigh.

While, analogously, no different than any other license violation, I predict that this finding will have a ripple effect in government adoption of open source projects in their applications, further dampening the adoption rates.

• Many government agencies are skeptical of open source in the first place.  Open source projects, rightly or wrongly, have a blanket aura of the two high school kids in a garage image.  Most government agencies like controls and process, which they don’t perceive exist in open source projects.  If some of them tried to become committers on major projects, perhaps the perception would change.
• Very few open source projects have sales representatives.  The sales representatives at major software companies such as Microsoft do an excellent job of the appropriate education and shepherding to provide assurances to the government buyers.
• Walking through a purchase process gives the perception of license compliance.  A contracting officer can say that they have made a purchase and have a license and feel like they are in compliance, although audits oftentimes reveal shortcomings.  With an open source project, anyone can download source code and deploy the software onto a system, creating manifold increases in risk of license violations.
• Most legal counsel isn’t familiar with open source licensing requirements and restrictions.  Rather than digging into and understanding GPL or “copy-left” restrictions, it’s easier to just stay away from it.  Again, there are probably as many, if not more, EULAs than accredited and accepted open source licenses, but there are also sales representatives to walk through EULAs whereas the open source projects do not have such champions.
• It’s easier to create blanket restrictions than to make judgments on each case based on the specific merits of the case.  The finding in Jacobsen v. Katzer makes it easier to point to a known and potentially quantifiable risk to deny use.  The Government is, and in most cases rightly so, risk-averse, and this finding creates a potentially big risk.  The risk may be a black swan, but it is very hard to measure and, in a perfect (in the Government’s view probably an imperfect) storm, could be potentially exceptionally large.

While it won’t have an immediate effect, the end result, I expect, will be an even more rigorous set of processes to complete for approval of the use of most open source software within the government.  I hope this is not the case, because there are many cases where open source software is the best solution.  Furthermore, I truly believe that:

• By and large, open source developers are not a litigious group.  They want to write software which makes life easier, and as long as they get the credit and attribution they want, they’re happy to share.  It’s what’s written in the licenses.  Plus, legal paths are expensive, and very few can afford to go down that path.  Compliance with a very few legal requirements or selection of a different platform will avoid any potential legal pratfalls.

Thanks to Onlyopensource for pointing out this article!

This move seems to be risky.  The Microsoft bet is that its FAST customers are more loyal to FAST than they are the operating platform, and the perception of switching costs are higher for moving from FAST to another enterprise search engine rather than the opposite–a loyalty to the operating system and a perception that search engines are interchangeable.

Microsoft might be right for most of its customers, but this announcement will certainly be grist for the mill in IT departments over the coming weeks.  Many companies built their IT infrastructure around a Linux-based platform, and being forced to change to a Windows environment may be a pill that is too hard to swallow.  The alternative will be to look to other search engines, which can do nothing but help Solr and Lucene.  With an established user base, enterprise grade support packages from companies like Lucid Imagination, and a significantly lower total cost of ownership than the FAST + Windows package, Solr will appeal to many a CTO who might otherwise have continued to gladly pay the licensing costs for FAST but is now forced to reconsider his or her decision.

Rather than supporting FAST on both platforms at the cost of a few developers, Microsoft may lose many more customers and revenues because of its insistence on one platform.  It will be interesting to see how companies like Lucid respond to the new opportunity.

While noble in trying to ensure that small businesses get compensated for work performed, the bill seems to miss the mark on the bigger issue with small business subcontracting.  I admit, I cannot find the text of the bill (and would love it if someone could point it out to me), so I could be wrong, but based on the article linked above, it appears that a couple of key issues are not addressed:

• Prime contractors actually meeting their subcontracting requirements.  In most large contracts, prime contractors are required to have a subcontracting plan and meet certain thresholds of work sharing amongst disadvantaged groups.  Very few enforcement mechanisms exist to ensure these thresholds are met, and this act does not address the issue.
• Prime contractors actually using the teams that they proposed in RFP responses.  If a prime wins a contract by describing teaming arrangements with certain companies, then it should actually utilize them as proposed to the government originally, or replace like-for-like on the team.  Again, no enforcement mechanism exists unless a contracting officer writes a constraining and binding contract.

As I have written before, set-asides and subcontracting goals go against my libertarian nature, as I want the most value for my money as a taxpayer; however, if they are going to exist, then the least the government can do is make sure that the rules are enforced.  Unfortunately, the Small Business Revitalization Act, it appears, fails to help meet those goals.

Have you ever wished that you could take Google with you on the road when you don’t have access to the INTRANET? or at least the documentation for whatever programming language you are currently using?

Groovy provides you with the option to download the documentation:

• as PDF (Very Old)
• as zipped HTML (Broken)

or does it?

2. Installation

So lets try installing Groovy on a Mac OS X:

• Download the source
• cd into the directory and run ‘ant install’
• simple right? not until you get this lovely message:

youssef-chakers-macbook:groovy-1-1.7.0 youssefchaker$ ant install
Buildfile: build.xml
Trying to override old definition of task javac

-excludeLegacyAntVersion:

-checkAntVersion:

-banner:
     [echo] Java Runtime Environment version: 1.5.0_22
     [echo] Java Runtime Environment vendor: Apple Inc.
     [echo] Ant version: Apache Ant version 1.7.1 compiled on June 27 2008
     [echo] Operating system name: Mac OS X
     [echo] Operating system architecture: i386
     [echo] Operating system version: 10.5.8
     [echo] Base directory: /opt/local/src/groovy-1-1.7.0
     [echo] Java Home: /System/Library/Frameworks/JavaVM.framework/Versions/1.5.0/Home

-initializeReports:
    [mkdir] Created dir: /opt/local/src/groovy-1-1.7.0/target/reports

-mavenTaskdef:

-mavenPomDefinitions:
     Processing /opt/local/src/groovy-1-1.7.0/pom.xml to /opt/local/src/groovy-1-1.7.0/target/groovy-all.pom
     Loading stylesheet /opt/local/src/groovy-1-1.7.0/config/maven/groovy-all.xsl

-mavenInit:

-mavenFetchAllModules:
[artifact:dependencies] Downloading: asm/asm/3.2/asm-3.2.pom from central
[artifact:dependencies] Downloading: asm/asm/3.2/asm-3.2.pom from central
[artifact:dependencies] Downloading: junit/junit/4.7/junit-4.7.pom from central
[artifact:dependencies] Downloading: junit/junit/4.7/junit-4.7.pom from central
[artifact:dependencies] Downloading: asm/asm-commons/3.2/asm-commons-3.2.pom from central
[artifact:dependencies] Downloading: asm/asm-commons/3.2/asm-commons-3.2.pom from central
[artifact:dependencies] Downloading: asm/asm-util/3.2/asm-util-3.2.pom from central
[artifact:dependencies] Downloading: asm/asm-util/3.2/asm-util-3.2.pom from central
[artifact:dependencies] Downloading: asm/asm-analysis/3.2/asm-analysis-3.2.pom from central
[artifact:dependencies] Downloading: asm/asm-analysis/3.2/asm-analysis-3.2.pom from central
[artifact:dependencies] Downloading: asm/asm-tree/3.2/asm-tree-3.2.pom from central
[artifact:dependencies] Downloading: asm/asm-tree/3.2/asm-tree-3.2.pom from central
[artifact:dependencies] Downloading: commons-logging/commons-logging/1.1.1/commons-logging-1.1.1.pom from central
[artifact:dependencies] Downloading: commons-logging/commons-logging/1.1.1/commons-logging-1.1.1.pom from central
[artifact:dependencies] Downloading: org/livetribe/livetribe-jsr223/2.0.6/livetribe-jsr223-2.0.6.pom from central
[artifact:dependencies] Downloading: org/livetribe/livetribe-jsr223/2.0.6/livetribe-jsr223-2.0.6.pom from central
[artifact:dependencies] Downloading: xmlunit/xmlunit/1.3/xmlunit-1.3.pom from central
[artifact:dependencies] Downloading: xmlunit/xmlunit/1.3/xmlunit-1.3.pom from central
[artifact:dependencies] Downloading: hsqldb/hsqldb/1.8.0.10/hsqldb-1.8.0.10.pom from central
[artifact:dependencies] Downloading: hsqldb/hsqldb/1.8.0.10/hsqldb-1.8.0.10.pom from central
[artifact:dependencies] Downloading: com/thoughtworks/xstream/xstream/1.3.1/xstream-1.3.1.pom from central
[artifact:dependencies] Downloading: com/thoughtworks/xstream/xstream/1.3.1/xstream-1.3.1.pom from central
[artifact:dependencies] Downloading: org/apache/ivy/ivy/2.1.0/ivy-2.1.0.pom from central
[artifact:dependencies] Downloading: org/apache/ivy/ivy/2.1.0/ivy-2.1.0.pom from central
[artifact:dependencies] Downloading: asm/asm/3.2/asm-3.2.jar from central
[artifact:dependencies] Downloading: asm/asm/3.2/asm-3.2.jar from central
[artifact:dependencies] Downloading: junit/junit/4.7/junit-4.7.jar from central
[artifact:dependencies] Downloading: junit/junit/4.7/junit-4.7.jar from central
[artifact:dependencies] Downloading: asm/asm-commons/3.2/asm-commons-3.2.jar from central
[artifact:dependencies] Downloading: asm/asm-commons/3.2/asm-commons-3.2.jar from central
[artifact:dependencies] Downloading: asm/asm-util/3.2/asm-util-3.2.jar from central
[artifact:dependencies] Downloading: asm/asm-util/3.2/asm-util-3.2.jar from central
[artifact:dependencies] Downloading: asm/asm-analysis/3.2/asm-analysis-3.2.jar from central
[artifact:dependencies] Downloading: asm/asm-analysis/3.2/asm-analysis-3.2.jar from central
[artifact:dependencies] Downloading: asm/asm-tree/3.2/asm-tree-3.2.jar from central
[artifact:dependencies] Downloading: asm/asm-tree/3.2/asm-tree-3.2.jar from central
[artifact:dependencies] Downloading: org/livetribe/livetribe-jsr223/2.0.6/livetribe-jsr223-2.0.6.jar from central
[artifact:dependencies] Downloading: org/livetribe/livetribe-jsr223/2.0.6/livetribe-jsr223-2.0.6.jar from central
[artifact:dependencies] Downloading: com/thoughtworks/xstream/xstream/1.3.1/xstream-1.3.1.jar from central
[artifact:dependencies] Downloading: com/thoughtworks/xstream/xstream/1.3.1/xstream-1.3.1.jar from central
[artifact:dependencies] Downloading: org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar from central
[artifact:dependencies] Downloading: org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar from central
[artifact:dependencies] An error has occurred while processing the Maven artifact tasks.
[artifact:dependencies]  Diagnosis:
[artifact:dependencies]
[artifact:dependencies] Unable to resolve artifact: Missing:
[artifact:dependencies] ----------
[artifact:dependencies] 1) asm:asm:jar:3.2
[artifact:dependencies]
[artifact:dependencies]   Try downloading the file manually from the project website.
[artifact:dependencies]
[artifact:dependencies]   Then, install it using the command:
[artifact:dependencies]       mvn install:install-file -DgroupId=asm -DartifactId=asm -Dversion=3.2 -Dpackaging=jar -Dfile=/path/to/file
[artifact:dependencies]
[artifact:dependencies]   Alternatively, if you host your own repository you can deploy the file there:
[artifact:dependencies]       mvn deploy:deploy-file -DgroupId=asm -DartifactId=asm -Dversion=3.2 -Dpackaging=jar -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]
[artifact:dependencies]
[artifact:dependencies]   Path to dependency:
[artifact:dependencies]         1) org.codehaus.groovy:groovy:jar:1.7.0
[artifact:dependencies]         2) asm:asm:jar:3.2
[artifact:dependencies]
[artifact:dependencies] 2) junit:junit:jar:4.7
[artifact:dependencies]
[artifact:dependencies]   Try downloading the file manually from the project website.
[artifact:dependencies]
[artifact:dependencies]   Then, install it using the command:
[artifact:dependencies]       mvn install:install-file -DgroupId=junit -DartifactId=junit -Dversion=4.7 -Dpackaging=jar -Dfile=/path/to/file
[artifact:dependencies]
[artifact:dependencies]   Alternatively, if you host your own repository you can deploy the file there:
[artifact:dependencies]       mvn deploy:deploy-file -DgroupId=junit -DartifactId=junit -Dversion=4.7 -Dpackaging=jar -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]
[artifact:dependencies]
[artifact:dependencies]   Path to dependency:
[artifact:dependencies]         1) org.codehaus.groovy:groovy:jar:1.7.0
[artifact:dependencies]         2) junit:junit:jar:4.7
[artifact:dependencies]
[artifact:dependencies] 3) asm:asm-commons:jar:3.2
[artifact:dependencies]
[artifact:dependencies]   Try downloading the file manually from the project website.
[artifact:dependencies]
[artifact:dependencies]   Then, install it using the command:
[artifact:dependencies]       mvn install:install-file -DgroupId=asm -DartifactId=asm-commons -Dversion=3.2 -Dpackaging=jar -Dfile=/path/to/file
[artifact:dependencies]
[artifact:dependencies]   Alternatively, if you host your own repository you can deploy the file there:
[artifact:dependencies]       mvn deploy:deploy-file -DgroupId=asm -DartifactId=asm-commons -Dversion=3.2 -Dpackaging=jar -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]
[artifact:dependencies]
[artifact:dependencies]   Path to dependency:
[artifact:dependencies]         1) org.codehaus.groovy:groovy:jar:1.7.0
[artifact:dependencies]         2) asm:asm-commons:jar:3.2
[artifact:dependencies]
[artifact:dependencies] 4) asm:asm-util:jar:3.2
[artifact:dependencies]
[artifact:dependencies]   Try downloading the file manually from the project website.
[artifact:dependencies]
[artifact:dependencies]   Then, install it using the command:
[artifact:dependencies]       mvn install:install-file -DgroupId=asm -DartifactId=asm-util -Dversion=3.2 -Dpackaging=jar -Dfile=/path/to/file
[artifact:dependencies]
[artifact:dependencies]   Alternatively, if you host your own repository you can deploy the file there:
[artifact:dependencies]       mvn deploy:deploy-file -DgroupId=asm -DartifactId=asm-util -Dversion=3.2 -Dpackaging=jar -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]
[artifact:dependencies]
[artifact:dependencies]   Path to dependency:
[artifact:dependencies]         1) org.codehaus.groovy:groovy:jar:1.7.0
[artifact:dependencies]         2) asm:asm-util:jar:3.2
[artifact:dependencies]
[artifact:dependencies] 5) asm:asm-analysis:jar:3.2
[artifact:dependencies]
[artifact:dependencies]   Try downloading the file manually from the project website.
[artifact:dependencies]
[artifact:dependencies]   Then, install it using the command:
[artifact:dependencies]       mvn install:install-file -DgroupId=asm -DartifactId=asm-analysis -Dversion=3.2 -Dpackaging=jar -Dfile=/path/to/file
[artifact:dependencies]
[artifact:dependencies]   Alternatively, if you host your own repository you can deploy the file there:
[artifact:dependencies]       mvn deploy:deploy-file -DgroupId=asm -DartifactId=asm-analysis -Dversion=3.2 -Dpackaging=jar -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]
[artifact:dependencies]
[artifact:dependencies]   Path to dependency:
[artifact:dependencies]         1) org.codehaus.groovy:groovy:jar:1.7.0
[artifact:dependencies]         2) asm:asm-analysis:jar:3.2
[artifact:dependencies]
[artifact:dependencies] 6) asm:asm-tree:jar:3.2
[artifact:dependencies]
[artifact:dependencies]   Try downloading the file manually from the project website.
[artifact:dependencies]
[artifact:dependencies]   Then, install it using the command:
[artifact:dependencies]       mvn install:install-file -DgroupId=asm -DartifactId=asm-tree -Dversion=3.2 -Dpackaging=jar -Dfile=/path/to/file
[artifact:dependencies]
[artifact:dependencies]   Alternatively, if you host your own repository you can deploy the file there:
[artifact:dependencies]       mvn deploy:deploy-file -DgroupId=asm -DartifactId=asm-tree -Dversion=3.2 -Dpackaging=jar -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]
[artifact:dependencies]
[artifact:dependencies]   Path to dependency:
[artifact:dependencies]         1) org.codehaus.groovy:groovy:jar:1.7.0
[artifact:dependencies]         2) asm:asm-tree:jar:3.2
[artifact:dependencies]
[artifact:dependencies] 7) org.livetribe:livetribe-jsr223:jar:2.0.6
[artifact:dependencies]
[artifact:dependencies]   Try downloading the file manually from the project website.
[artifact:dependencies]
[artifact:dependencies]   Then, install it using the command:
[artifact:dependencies]       mvn install:install-file -DgroupId=org.livetribe -DartifactId=livetribe-jsr223 -Dversion=2.0.6 -Dpackaging=jar -Dfile=/path/to/file
[artifact:dependencies]
[artifact:dependencies]   Alternatively, if you host your own repository you can deploy the file there:
[artifact:dependencies]       mvn deploy:deploy-file -DgroupId=org.livetribe -DartifactId=livetribe-jsr223 -Dversion=2.0.6 -Dpackaging=jar -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]
[artifact:dependencies]
[artifact:dependencies]   Path to dependency:
[artifact:dependencies]         1) org.codehaus.groovy:groovy:jar:1.7.0
[artifact:dependencies]         2) org.livetribe:livetribe-jsr223:jar:2.0.6
[artifact:dependencies]
[artifact:dependencies] 8 ) com.thoughtworks.xstream:xstream:jar:1.3.1
[artifact:dependencies]
[artifact:dependencies]   Try downloading the file manually from the project website.
[artifact:dependencies]
[artifact:dependencies]   Then, install it using the command:
[artifact:dependencies]       mvn install:install-file -DgroupId=com.thoughtworks.xstream -DartifactId=xstream -Dversion=1.3.1 -Dpackaging=jar -Dfile=/path/to/file
[artifact:dependencies]
[artifact:dependencies]   Alternatively, if you host your own repository you can deploy the file there:
[artifact:dependencies]       mvn deploy:deploy-file -DgroupId=com.thoughtworks.xstream -DartifactId=xstream -Dversion=1.3.1 -Dpackaging=jar -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]
[artifact:dependencies]
[artifact:dependencies]   Path to dependency:
[artifact:dependencies]         1) org.codehaus.groovy:groovy:jar:1.7.0
[artifact:dependencies]         2) com.thoughtworks.xstream:xstream:jar:1.3.1
[artifact:dependencies]
[artifact:dependencies] 9) org.apache.ivy:ivy:jar:2.1.0
[artifact:dependencies]
[artifact:dependencies]   Try downloading the file manually from the project website.
[artifact:dependencies]
[artifact:dependencies]   Then, install it using the command:
[artifact:dependencies]       mvn install:install-file -DgroupId=org.apache.ivy -DartifactId=ivy -Dversion=2.1.0 -Dpackaging=jar -Dfile=/path/to/file
[artifact:dependencies]
[artifact:dependencies]   Alternatively, if you host your own repository you can deploy the file there:
[artifact:dependencies]       mvn deploy:deploy-file -DgroupId=org.apache.ivy -DartifactId=ivy -Dversion=2.1.0 -Dpackaging=jar -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]
[artifact:dependencies]
[artifact:dependencies]   Path to dependency:
[artifact:dependencies]         1) org.codehaus.groovy:groovy:jar:1.7.0
[artifact:dependencies]         2) org.apache.ivy:ivy:jar:2.1.0
[artifact:dependencies]
[artifact:dependencies] ----------
[artifact:dependencies] 9 required artifacts are missing.
[artifact:dependencies]
[artifact:dependencies] for artifact:
[artifact:dependencies]   org.codehaus.groovy:groovy:jar:1.7.0
[artifact:dependencies]
[artifact:dependencies] from the specified remote repositories:
[artifact:dependencies]   central (http://repo1.maven.org/maven2)
[artifact:dependencies]
[artifact:dependencies] 

BUILD FAILED
/opt/local/src/groovy-1-1.7.0/config/ant/build-maven.xml:79: The following error occurred while executing this line:
/opt/local/src/groovy-1-1.7.0/config/ant/build-maven.xml:52: The following error occurred while executing this line:
/opt/local/src/groovy-1-1.7.0/config/ant/build-maven.xml:44: The following error occurred while executing this line:
/opt/local/src/groovy-1-1.7.0/config/ant/build-maven.xml:29: Unable to resolve artifact: Missing:
----------
1) asm:asm:jar:3.2

  Try downloading the file manually from the project website.

  Then, install it using the command:
      mvn install:install-file -DgroupId=asm -DartifactId=asm -Dversion=3.2 -Dpackaging=jar -Dfile=/path/to/file

  Alternatively, if you host your own repository you can deploy the file there:
      mvn deploy:deploy-file -DgroupId=asm -DartifactId=asm -Dversion=3.2 -Dpackaging=jar -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]

  Path to dependency:
        1) org.codehaus.groovy:groovy:jar:1.7.0
        2) asm:asm:jar:3.2

2) junit:junit:jar:4.7

  Try downloading the file manually from the project website.

  Then, install it using the command:
      mvn install:install-file -DgroupId=junit -DartifactId=junit -Dversion=4.7 -Dpackaging=jar -Dfile=/path/to/file

  Alternatively, if you host your own repository you can deploy the file there:
      mvn deploy:deploy-file -DgroupId=junit -DartifactId=junit -Dversion=4.7 -Dpackaging=jar -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]

  Path to dependency:
        1) org.codehaus.groovy:groovy:jar:1.7.0
        2) junit:junit:jar:4.7

3) asm:asm-commons:jar:3.2

  Try downloading the file manually from the project website.

  Then, install it using the command:
      mvn install:install-file -DgroupId=asm -DartifactId=asm-commons -Dversion=3.2 -Dpackaging=jar -Dfile=/path/to/file

  Alternatively, if you host your own repository you can deploy the file there:
      mvn deploy:deploy-file -DgroupId=asm -DartifactId=asm-commons -Dversion=3.2 -Dpackaging=jar -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]

  Path to dependency:
        1) org.codehaus.groovy:groovy:jar:1.7.0
        2) asm:asm-commons:jar:3.2

4) asm:asm-util:jar:3.2

  Try downloading the file manually from the project website.

  Then, install it using the command:
      mvn install:install-file -DgroupId=asm -DartifactId=asm-util -Dversion=3.2 -Dpackaging=jar -Dfile=/path/to/file

  Alternatively, if you host your own repository you can deploy the file there:
      mvn deploy:deploy-file -DgroupId=asm -DartifactId=asm-util -Dversion=3.2 -Dpackaging=jar -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]

  Path to dependency:
        1) org.codehaus.groovy:groovy:jar:1.7.0
        2) asm:asm-util:jar:3.2

5) asm:asm-analysis:jar:3.2

  Try downloading the file manually from the project website.

  Then, install it using the command:
      mvn install:install-file -DgroupId=asm -DartifactId=asm-analysis -Dversion=3.2 -Dpackaging=jar -Dfile=/path/to/file

  Alternatively, if you host your own repository you can deploy the file there:
      mvn deploy:deploy-file -DgroupId=asm -DartifactId=asm-analysis -Dversion=3.2 -Dpackaging=jar -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]

  Path to dependency:
        1) org.codehaus.groovy:groovy:jar:1.7.0
        2) asm:asm-analysis:jar:3.2

6) asm:asm-tree:jar:3.2

  Try downloading the file manually from the project website.

  Then, install it using the command:
      mvn install:install-file -DgroupId=asm -DartifactId=asm-tree -Dversion=3.2 -Dpackaging=jar -Dfile=/path/to/file

  Alternatively, if you host your own repository you can deploy the file there:
      mvn deploy:deploy-file -DgroupId=asm -DartifactId=asm-tree -Dversion=3.2 -Dpackaging=jar -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]

  Path to dependency:
        1) org.codehaus.groovy:groovy:jar:1.7.0
        2) asm:asm-tree:jar:3.2

7) org.livetribe:livetribe-jsr223:jar:2.0.6

  Try downloading the file manually from the project website.

  Then, install it using the command:
      mvn install:install-file -DgroupId=org.livetribe -DartifactId=livetribe-jsr223 -Dversion=2.0.6 -Dpackaging=jar -Dfile=/path/to/file

  Alternatively, if you host your own repository you can deploy the file there:
      mvn deploy:deploy-file -DgroupId=org.livetribe -DartifactId=livetribe-jsr223 -Dversion=2.0.6 -Dpackaging=jar -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]

  Path to dependency:
        1) org.codehaus.groovy:groovy:jar:1.7.0
        2) org.livetribe:livetribe-jsr223:jar:2.0.6

8 ) com.thoughtworks.xstream:xstream:jar:1.3.1

  Try downloading the file manually from the project website.

  Then, install it using the command:
      mvn install:install-file -DgroupId=com.thoughtworks.xstream -DartifactId=xstream -Dversion=1.3.1 -Dpackaging=jar -Dfile=/path/to/file

  Alternatively, if you host your own repository you can deploy the file there:
      mvn deploy:deploy-file -DgroupId=com.thoughtworks.xstream -DartifactId=xstream -Dversion=1.3.1 -Dpackaging=jar -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]

  Path to dependency:
        1) org.codehaus.groovy:groovy:jar:1.7.0
        2) com.thoughtworks.xstream:xstream:jar:1.3.1

9) org.apache.ivy:ivy:jar:2.1.0

  Try downloading the file manually from the project website.

  Then, install it using the command:
      mvn install:install-file -DgroupId=org.apache.ivy -DartifactId=ivy -Dversion=2.1.0 -Dpackaging=jar -Dfile=/path/to/file

  Alternatively, if you host your own repository you can deploy the file there:
      mvn deploy:deploy-file -DgroupId=org.apache.ivy -DartifactId=ivy -Dversion=2.1.0 -Dpackaging=jar -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]

  Path to dependency:
        1) org.codehaus.groovy:groovy:jar:1.7.0
        2) org.apache.ivy:ivy:jar:2.1.0

----------
9 required artifacts are missing.

for artifact:
  org.codehaus.groovy:groovy:jar:1.7.0

from the specified remote repositories:
  central (http://repo1.maven.org/maven2)

Total time: 18 seconds

Ok, how about we try MacPorts?

youssef-chakers-macbook:~ youssefchaker$ sudo port install groovyPassword:--->  Fetching apache-ant--->  Attempting to fetch apache-ant-1.7.1-bin.tar.bz2 from http://www.ibiblio.org/pub/mirrors/apache/ant/binaries--->  Attempting to fetch apache-ant-1.7.1-bin.tar.bz2 from http://apache.mirror.rafal.ca/ant/binaries--->  Attempting to fetch apache-ant-1.7.1-bin.tar.bz2 from http://apache.adcserver.com.ar/ant/binaries--->  Attempting to fetch apache-ant-1.7.1-bin.tar.bz2 from http://www.mirrorservice.org/sites/ftp.apache.org/ant/binaries--->  Attempting to fetch apache-ant-1.7.1-bin.tar.bz2 from http://apache.mirroring.de/ant/binaries--->  Attempting to fetch apache-ant-1.7.1-bin.tar.bz2 from http://apache.multidist.com/ant/binaries--->  Attempting to fetch apache-ant-1.7.1-bin.tar.bz2 from ftp://ftp.infoscience.co.jp/pub/net/apache/dist/ant/binaries--->  Verifying checksum(s) for apache-ant--->  Extracting apache-ant--->  Configuring apache-ant--->  Building apache-ant--->  Staging apache-ant into destroot--->  Installing apache-ant @1.7.1_0--->  Activating apache-ant @1.7.1_0--->  Cleaning apache-ant--->  Fetching groovy--->  Attempting to fetch groovy-src-1.6.3.zip from http://distfiles.macports.org/groovy--->  Verifying checksum(s) for groovy--->  Extracting groovy--->  Configuring groovy--->  Building groovyError: Target org.macports.build returned: shell command " cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_java_groovy/work/groovy-1.6.3" && ant install -DskipTests=true " returned error 1Command output: /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_java_groovy/work/groovy-1.6.3/config/ant/build-maven.xml:90: The following error occurred while executing this line:/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_java_groovy/work/groovy-1.6.3/config/ant/build-maven.xml:54: The following error occurred while executing this line:/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_java_groovy/work/groovy-1.6.3/config/ant/build-maven.xml:29: Unable to resolve artifact: Missing:----------1) biz.aQute:bnd:jar:0.0.258Try downloading the file manually from the project website.Then, install it using the command:mvn install:install-file -DgroupId=biz.aQute -DartifactId=bnd -Dversion=0.0.258 -Dpackaging=jar -Dfile=/path/to/fileAlternatively, if you host your own repository you can deploy the file there:mvn deploy:deploy-file -DgroupId=biz.aQute -DartifactId=bnd -Dversion=0.0.258 -Dpackaging=jar -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]Path to dependency:1) groovy:groovy-tools:jar:internal2) biz.aQute:bnd:jar:0.0.258----------1 required artifact is missing.for artifact:groovy:groovy-tools:jar:internalfrom the specified remote repositories:central (http://repo1.maven.org/maven2),aQute (http://www.aQute.biz/repo)Total time: 2 minutes 51 secondsError: Status 1 encountered during processing.
youssef-chakers-macbook:~ youssefchaker$ sudo port install groovy

Password:

--->  Fetching apache-ant

--->  Attempting to fetch apache-ant-1.7.1-bin.tar.bz2 from http://www.ibiblio.org/pub/mirrors/apache/ant/binaries

--->  Attempting to fetch apache-ant-1.7.1-bin.tar.bz2 from http://apache.mirror.rafal.ca/ant/binaries

--->  Attempting to fetch apache-ant-1.7.1-bin.tar.bz2 from http://apache.adcserver.com.ar/ant/binaries

--->  Attempting to fetch apache-ant-1.7.1-bin.tar.bz2 from http://www.mirrorservice.org/sites/ftp.apache.org/ant/binaries

--->  Attempting to fetch apache-ant-1.7.1-bin.tar.bz2 from http://apache.mirroring.de/ant/binaries

--->  Attempting to fetch apache-ant-1.7.1-bin.tar.bz2 from http://apache.multidist.com/ant/binaries

--->  Attempting to fetch apache-ant-1.7.1-bin.tar.bz2 from ftp://ftp.infoscience.co.jp/pub/net/apache/dist/ant/binaries

--->  Verifying checksum(s) for apache-ant

--->  Extracting apache-ant

--->  Configuring apache-ant

--->  Building apache-ant

--->  Staging apache-ant into destroot

--->  Installing apache-ant @1.7.1_0

--->  Activating apache-ant @1.7.1_0

--->  Cleaning apache-ant

--->  Fetching groovy

--->  Attempting to fetch groovy-src-1.6.3.zip from http://distfiles.macports.org/groovy

--->  Verifying checksum(s) for groovy

--->  Extracting groovy

--->  Configuring groovy

--->  Building groovy

Error: Target org.macports.build returned: shell command " cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_java_groovy/work/groovy-1.6.3" && ant install -DskipTests=true " returned error 1

Command output: /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_java_groovy/work/groovy-1.6.3/config/ant/build-maven.xml:90: The following error occurred while executing this line:

/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_java_groovy/work/groovy-1.6.3/config/ant/build-maven.xml:54: The following error occurred while executing this line:

/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_java_groovy/work/groovy-1.6.3/config/ant/build-maven.xml:29: Unable to resolve artifact: Missing:

----------

1) biz.aQute:bnd:jar:0.0.258

Try downloading the file manually from the project website.

Then, install it using the command:

mvn install:install-file -DgroupId=biz.aQute -DartifactId=bnd -Dversion=0.0.258 -Dpackaging=jar -Dfile=/path/to/file

Alternatively, if you host your own repository you can deploy the file there:

mvn deploy:deploy-file -DgroupId=biz.aQute -DartifactId=bnd -Dversion=0.0.258 -Dpackaging=jar -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]

Path to dependency:

1) groovy:groovy-tools:jar:internal

2) biz.aQute:bnd:jar:0.0.258

----------

1 required artifact is missing.

for artifact:

groovy:groovy-tools:jar:internal

from the specified remote repositories:

central (http://repo1.maven.org/maven2),

aQute (http://www.aQute.biz/repo)

Total time: 2 minutes 51 seconds

Error: Status 1 encountered during processing.

meh!

sudo apt-get install groovy

to the rescue : D

3. Compiling!

I attended a presentation by Andy Hunt, he was talking about some of the stuff that he wrote in his book “Pragmatic Thinking & Leaning”, which is very good by the way, and he mentioned context switching a few times. Naturally, that came up in often afterwards, specially on twitter and I brushed it off mostly. I did some context switching but only when taking breaks and I didn’t think it was a big deal. Not until I saw this line:

“[groovyc] Compiling 1 source file …”

and it hit me. Most people are tempted to do some context switching when they are dealing with time wasting tasks such as compiling! I had completely forgotten how it felt like. I was so immersed in the Ruby/Python/PHP world that I totally erased the memories of “It’s compiling…” from my brain!!!

But in all seriousness, if you like Java but also appreciate what a language like Ruby provides and what a framework like Ruby on Rails adds to the table, Groovy and Grails are worth a shot. And if you want a quick overview and introduction to both, check out my friend, Mohamed Seifeddine’s thesis:

This document is written as my Masters’ thesis and is based on a long individual learning period of Groovy and Grails through the developement of what later became SoukLubnan.com. A Website in its younger days that allows people in Lebanon to Buy & Sell products online.

and here’s the link: Introduction to Groovy and Grails

Installer

The first thing you’ll notice is that an installer (and uninstaller) is provided that walks you through the basic steps of installing Solr. Now Solr itself is pretty darn simple to work with already, but you do need to compile the code, which means you need Ant to be installed. The Lucid installer avoids that need, and  adds support for running Solr in Tomcat as well as Jetty. And, assuming you have a support agreement with Lucid, it supports downloading plugins from Lucid to extend your Solr platform. Right now the only free plugin is the Reference Guide PDF. Having an installer available definitely checks a box for the systems type folks who may be installing Solr, but it doesn’t really do anything crazy special. Also, one nit is that if you install into /opt/dirA, and then want to install into /opt/dirB, you have to delete ~/.LucidWorks/ directory as the install dir is cached!  But it does demonstrate what might be coming from Lucid in future updates!

Installer Targets Screen

Another enhancment from Lucid is a Tray Application for managing your Solr instances. However, this turns out to just be a basic (on OSX at least!) menubar application that allows you to start/stop a local Solr server. There doesn’t seem to be any options to stop and start remote servers, or monitor the health of running Solrs, so I think this is something you use once and never again! Hey Lucid, it would be great though if the Tray App integrated stoplight monitoring of Solr instances and popped open web pages to admin pages to perform various tasks on your collection of Solr servers!

Directory Layout

The directory that you’ve installed Solr into should look very familiar. In fact, too familiar to me! I’ve gone back and forth on the way that Solr is distributed with source code as well as compiled jars. While Solr used to be a tool that only Java centric shops would look at, it’s now gone mainstream, to where many, if not most, organizations that use Solr are not traditional Java shops! I really wish I could download a version of Solr that didn’t have the src directory, was just a stripped down ready to go application. Admittedly, the example application that is part of the source functions as a template, but it has been bemoaned by myself and others that folks just use and abuse the configuration of what was meant as an example app, to their detriment!

So I was hoping that the LucidWorks distros’ Installer would function as that smart template by walking me through including/excluding various extensions like DIH, Clustering, and Extraction. But at least in this first version, no such luck. The support though for for picking either Tomcat or Jetty as a container shows what could be in the offing though!

While the LucidWorks distro still ships with the hoary old example directory is still there, there is now a lucidworks directory. When you run the new toplevel start.sh shell script it starts Solr with solr.solr.home=lucidworks/solr directory. Something to note is that the start.sh has complete paths defined in it from the installer:

cd /Users/epugh/solr/solr2/LucidWorks/lucidworks/jetty/../

It really should at least have a single variable at the top that you can changing depending on what environment you are in.

The lucidworks project is also setup as a single index project.  Since the future is multicore configurations, I’d like to see that as the default in more examples.  (The example app needs a bit of work as well to better show off multicore as a first class feature!)

solrconfig.xml

Doing a diff on the example and lucidworks versions of solrconfig.xml shows its pretty much the same as the one from the example app, but with the correct configurations for DataImportHandler and the Velocity based search UI called Solritas. Solritas is a nice tool for helping you “wedge” Solr into places by providing a simple Velocity template based translation layer, and even build a GUI, within your Solr environment. Solritas hasn’t received a lot of buzz, so it’s nice seeing it turned on by default! The clustering functionality is also specified, but not sure if the solr.cluster.enabled=true startup parameter is actually required or not.

The other oddity is that the Lucid monitoring product for Solr, SolrGaze, isn’t enabled by default! Doesn’t seem like the most ringing endorsement for the software. I’m excited by the prospect of better visiblity into the internals of Solr, so I enabled it.

schema.xml

Diffing the two schema.xml files reveals the addition of the Lucid KStemmer com.lucidimagination.solrworks.analysis.LucidKStemFilterFactory for fast non-aggresive text stemming. According to Lucid it provides:

Large field performance shows a 220% performance increase, while small fields show a 1140% increase compared to the original UMASS code.

SolrGaze

SolrGaze promises to make it easier to see what is going on inside of Solr. Anything that makes it simpler for operations folks instead of developers to manage Solr is good in my book. I ran into one nit which was I opened up SolrGaze using the url http://localhost:8983/gaze/index.html. It barfed connecting to Solr to display gathered metrics, but if I used http://127.0.0.1:8983/gaze/index.html then everything was fine.

I haven’t had to chance to really play with Gaze yet, so I’ll post a more in-depth review soon.

Summary

All in all, the Lucid distro would be what I would recommend for a first timer to download, or someone doing a spike of development and needing a quick install of Solr.  Not requiring Ant to be installed is a wonderful thing, and being pre-configured for Clustering, DIH, and Solritas means you get to see a working Solr install, complete with a full featured GUI, right out of the box.  In terms of using for a production deploy, there is less to recommend it since you’re going to want to strip down to just the bits and bobs that your require for your specific needs.  I haven’t delved down into what SolrGaze provides, so that feature may be the tipping point for deciding to use the Lucid distribution.

He has written a very indepth review, that I’ll admit I was nervous to read! But he summed it up as:

Grand Finale
I spelled out a lot of fiddly feedback above, and I expect the great addendum wiki page will factor in any keepers from this review. Of course most of the review points out mistakes or differences of opinion, that’s what a review is for, though this is a solid, useful book. So, if you’re considering using Solr, this book is for you. If you’re already using Solr, you’ll likely pick up a useful trick or three. Go get it!

As you can see from the level of detail in his post, when we come out with a second version of the Solr book, updating it for changes between when we published it and the final release of Solr 1.4 will be very easy!

Today I started exploring BrowserCMS and have had a good day playing around with it. Check out the details in this post.