There are times, when I am investigating a problem, it is helpful to turn on DEBUG logging for the LabKey Server. As you would expect, enabling DEBUG logging, produces LOTS of log entries, so it is not something that you want to keep enabled forever, but it can be helpful to enable while debugging.

There are a number of ways to enable DEBUG logging. The easiest will simply flood the labkey.loglog file with the DEBUG messages, which is not ideal. I like to write the DEBUG logs to a separate, new log file.

The LabKey Server uses log4j for logging. The configuration file is located at LABKEY_HOME/labkeywebapp/WEB-INF/classes/log4j.xml, where LABKEY_HOME is the installation directory for your LabKey Server.

With the changes documented below, the new logging behavior will be

1. The log file, labkey-errors.log will continue to receive all LabKey-specific messages with priority of ERROR or greater
2. The log file, labkey.log will continue to receive all LabKey-specific messages with priority of INFO or greater
3. The new log file, labkey-debug.log will receive all LabKey-specific messages with priority of DEBUG or greater

Please be warned, the labkey-debug.log file can grow very fast. The configuration below sets the max log file size to 10MB. When the file size reaches 10MB, it will be automatically rollover to a new file. Four previous log files will be retained.

Enable DEBUG level logging

You can follow these instructions to enable DEBUG level logging for your LabKey Server

1) Stop your LabKey Server

2) Make a backup of the existing log4j configuration file

cd LABKEY_HOME/labkeywebapp/WEB-INF/classes/log4j.xml
cp log4j.xml log4j.xml.original

3) Edit log4j.xml file

4) Start your LabKey Server

Now if you go directory which holds the log files, CATALINA_HOME/logs you should see a file named labkey-debug.log.

Once you are done debugging your problem, you can disable the DEBUG logging by

1) Stop your LabKey Server

2) Restore the backup of the log4j configuration file

cd LABKEY_HOME/labkeywebapp/WEB-INF/classes/log4j.xml
cp log4j.xml.original log4j.xml

3) Start your LabKey Server

Using the DEBUG log file.

labkey-debug.log will contain lots and lots of DEBUG messages. The best way to use the new log file, is to use a tool like grep to only output the messages you are interested in. For example, if you are interested in tracking the progress of the deletion of a Project, you can use

tail -f labkey-debug.log | grep ContainerManager

or you see all messages about the deletion by running

less labkey-debug* | grep ContainerManager

Below you will find instructions for using LabKey’s sample database backup script to backup your PostgreSQL databases on Windows. This is a very simple script which backups up the databases to the specified directory. Please feel free to use it as a starting point for backing up the database for your LabKey Server.

What will this script do

This script can be used to perform a nightly backup (using pg_dump) of databases on a PostgreSQL server instance. The script will

1. Perform a full backup of the postgres and labkey databases using pg_dump

2. Dump archives will be placed in the %BACKUP_DIR% directory

3. Dump archive file names will be in the format postgresql-backup-DB_YYYYMMDD.bak where

   • DB is the database name
   • YYYY is the year
   • MM is the month
   • DD is the day of the month

4. Write status and error messages to STDOUT

5. Write status and error messages to the logfile (%LOGFILE%)

Install the backup script

Install the sample backup script in a directory, such as c:\labkey\bin, on the server running your PostgreSQL database instance. In many cases, this will be the same server running your LabKey Server.

Customize the backup script for your server

You will need to customize the script for your server and environment.

1. Change the name of the script to something more meaningful to you, such as labkey-database-backup.bat

2. Open the script and edit the variables as needed. The important variables are

   • BACKUP_DIR: This is the directory where the backups will be stored
   • POSTGRES_HOME: The directory where your PostgreSQL database is installed
   • POSTGRES_USER: The user account to be used for the backup.
   • PGPASSFILE: Location of the pgpass file which will be used for authenticating to the PostgreSQL server

Create the pgpass file

To authenticate to the PostgreSQL database server, the script will use a pgpass file. This allows us to not have the database password in clear-text in the backup script.

1. Create a pgpass file and place it in a secure location.

   • For example, you could place it in your HOME directory at C:\Users\ACCOUNTNAME\postgresql\pgpass.conf
   • where ACCOUNTNAME is your username.

2. Add the login credentials to the pgpass file

   • The format of the pgpass is

       hostname:port:database:username:password

   • For example, if you plan on performing the backup using the postgres database user, the file may contain

       localhost:5432:postgres:postgres:THEPASSWORD
       localhost:5432:labkey:postgres:THEPASSWORD

3. Secure the pgpass file

   • Using NTFS permissions, disable access to this file for everybody except for the ACCOUNTNAME user

4. Edit the backup script and change the PGPASSFILE variable to be path to the file created in step 1.

Running the backup script.

To execute the backup script, you will need to open a command prompt and run

C:\labkey\bin\labkey-database-backup.bat

NOTE: This example assumes you have installed the backup script at C:\labkey\bin\labkey-database-backup.bat

The script will output status and error messages to the screen. In addition, it will write these messages, plus additional debug messages to a log file located at %BACKUP_DIR%\labkey-database-backup.log

Create a scheduled task OPTIONAL:

If you would like to perform a backup on a nightly basis, you can create scheduled task in windows.

The command to use is

C:\Windows\System32\cmd.exe /c "C:\labkey\bin\labkey-database-backup.bat"

The start directory to use will be the directory where the backup script is located (for example C:\labkey\bin)

NOTE: This example assumes you have installed the backup script at C:\labkey\bin\labkey-database-backup.bat

With the recent PostgreSQL security update, I was forced to perform emergency maintenance on all my PostgreSQL servers to install the update. To remind myself of the process, I did a quick search and it turns out I could not find great instructions on installing “Point Releases” of PostgreSQL.

Here are my step by step instructions, for the next time I need to do this.

At LabKey, we run lots of different versions of PostgreSQL(for testing, etc) and I always install PostgreSQL from source. The instructions below will be for upgrading v9.0.

One of the great things about upgrading to a new point release, is that you are not required perform the usual methodology of

1. Backup database
2. Stop database server
3. Install new version of database
4. Restore database backup from Step #1

You can simply install the new version and start the server up and it should work. The methodology I used is

1. Backup the databases

[Optional] For safe keeping, backup the database using pg_dump

/usr/local/postgresql-9.0/bin/pg_dumpall -U postgres -W -f /backup/postgresql_backup_for_upgrade_to_9.0.13.out

2. Build the new version

sudo su - 
cd /usr/local/src
wget http://ftp.postgresql.org/pub/source/v9.0.13/postgresql-9.0.13.tar.gz
tar xzf postgresql-9.0.13.tar.gz
cd postgresql-9.0.13/
./configure --prefix=/usr/local/postgresql-9.0/
make

3. Shutdown PostgreSQL server

/etc/init.d/postgresql-9.0 stop

Backup the installation

For safe keeping, make a backup of the installed binaries, etc

If you have small databases or your PGDATA directory is not located in the PostgreSQL install directory , you can run

cd /usr/local
cp -Rp postgresql-9.0 postgresql-9.0_backup

If your databases are large, then you can run

cd /usr/local
tar --exclude="./postgresql-9.0/data" -czf postgresql_backup_for_upgrade_to_9.0.13.tar.gz postgresql-9.0 

Upgrade PostgreSQL to 9.0.13 and start the server

cd /usr/local/src/postgresql-9.0.13/
make install 
/etc/init.d/postgresql-9.0 start

That is it. All done.

At LabKey, I run into this problem over and over again. I assist a customer migrate their Proteomics pipeline to new hardware (and a newer linux distribution) and none of the older Proteomics tools compile properly. I received an email from a customer, yesterday asking for assistance in compiling PepMatch. The customer was trying to compile the software on RHEL6.2. They were getting the error

[labkey@server pepmatch]$ make
...
g++ -O2 -DGCC   -c -o InspectResultsParser.o InspectResultsParser.cpp
In file included from PepResultsParser.h:4,
                 from InspectResultsParser.cpp:7:
saxhandler.h: In member function ‘bool SAXHandler::isElement(const char*, const XML_Char*)’:
saxhandler.h:61: error: ‘strcmp’ was not declared in this scope
saxhandler.h: In member function ‘bool SAXHandler::isAttr(const char*, const XML_Char*)’:
saxhandler.h:64: error: ‘strcmp’ was not declared in this scope
InspectResultsParser.cpp: In function ‘char* prepareAppend(char*)’:
InspectResultsParser.cpp:12: error: ‘strlen’ was not declared in this scope
InspectResultsParser.cpp: In member function ‘bool InspectResultsParser::parse()’:
InspectResultsParser.cpp:66: error: ‘strstr’ was not declared in this scope
InspectResultsParser.cpp:118: error: ‘strcasecmp’ was not declared in this scope
make: *** [InspectResultsParser.o] Error 1

 
I vaguely remembered this occurring before, but could not find anything in my notes or email about it. After a bit of googling, I found GCC 4.3 Release Series Porting to the New Tools, which says,

As detailed here (Header dependency streamlining), many of the standard C++ library include files have been edited to only include the smallest possible number of additional files. As such, many C++ programs that used std::memcpy without including <cstring>, or used std::auto_ptr without including <memory> will no longer compile.

 

Usually, this error is of the form:

 

error: 'strcmp' was not declared in this scope

 
Sure enough, I added #include <cscript> to the saxhandler.h file and the compile successfully completed.

I posted this here so I can find it the next time either I or a customer experience a problem like this.

LabKey uses Google Apps for email, calendar and document sharing. With the release of Google Drive, I got a request to sync some files from an internal file share to the Google Doc/Drive.

The file server is on a linux server(Ubuntu 10.04 LTS), running SAMBA. There is currently not a linux client for Google Drive. But there is an open source client, Grive that I used for testing.

Given that Ubuntu 10.04 is getting a little long in the tooth, it took a bit of effort to get Grive compiled on the server.

Install the required software

There is not whole lot of documentation, but the Grive webpage gives an overview of the required libraries. I started from there and installed

apt-get install build-essential cmake make openssl
apt-get install libstdc++6 libstdc++6-4.4-dev
apt-get install libjson0 libjson0-dev expat libexpat1-dev
apt-get install libssl-dev libssl0.9.8
apt-get install libcurl4-openssl-dev
apt-get install binutils-dev

Install BOOST

The BOOST libraries are required, however, the compile failed when using v1.4.0, which is installed via APT. After some digging, I found that compilation problems were fixed in later versions. I downloaded and installed BOOST 1.4.9.

Install the perquisite software

apt-get install bzip2 libbz2-dev zip unzip 
apt-get install python-dev

Download the latest version of BOOST

cd /usr/local/src 
wget http://softlayer.dl.sourceforge.net/project/boost/boost/1.49.0/boost_1_49_0.tar.gz
tar xzf boost_1_49_0.tar.gz

Build and install the BOOST libraries

./bootstrap.sh
./b2
./b2 install

This installs the libraries into /usr/local/lib and the header files into /usr/local/include

Build the grive software

Download and expand the software distribution

cd /usr/local/src
wget https://github.com/match065/grive/tarball/master
mv master match065-grive-b6fb4a6.tar.gz
tar xzf match065-grive-b6fb4a6.tar.gz

Build and install the software

cmake CMakeLists.txt
make 
make install 

This installs the binary at /usr/local/bin/grive and the libraries into /usr/local/lib

Using grive software

In order to use the grive software, you will need to set the LD_LIBRARY_PATH to include /usr/local/lib. ie something like

export LD_LIBRARY_PATH=/usr/local/lib

Initialize

1. Create a directory to hold the files from your Google Drive
2. Follow the instructions in the Usage section at the Grive webpage to grant Grive access to your Google Drive.

NOTE: The authorization credentials for accessing your Google Drive account are stored in a file named .grive. I strongly recommend

• securing this file so it can only be read by the owner of the directory
• blocking access to this file via SAMBA, if this is going to be a shared drive. (See hide files option)

Syncing your files and directories

To sync your files form Google Drive down to the local directory, all you need to do it run

/usr/local/bin/grive 

Grive will sync in both direction and seems to work great. This is not a daemon, you will need to manually perform the sync after any files have been changed or your could use a CRON job to periodically perform the sync.

When using Tomcat6 on a server/desktop that is running both IPv4 and IPv6 networks, there are times when the Tomcat connector will bind to either the IPv4 or IPv6 interface and not both. This only occurs on kernels where the IPv4 and IPv6 stack do not share a common listener. Personally, I have had this occur on Windows7, Windows Server 2008, and RHEL6 servers.

There are some tricks to get around this.

Trick #1:

Add the following to each connector in your server.xml file.

address="0.0.0.0"

This configuration works reliably on Windows servers, but on RHEL 6.x this does not work

Trick #2:

This one I found buried in a blog entry. Add the following to your CATALINA_OPTS variable in your startup script.

-Djava.net.preferIPv4Stack=true

This worked on RHEL6.

LabKey as a company was started by a bunch of developers and is still run by these same developers. If you asked then how they want to manage tasks, their first response will be in the bug list. For some of our business related tasks, instead of fighting this, we decided to embrace it. To embrace the bug list, we needed a way to pragmatically create new bugs(Issues) from an existing business system.

A problem we had to overcome was how to programmatically create new Issues. LabKey Server currently does not have a client API for creating or managing Issues, so I had to do it the old fashion way.

Creating new LabKey Server Issues with Python

The first thing we have to do is authenticate to the server (See this blog post for more information on authenticating to LabKey Server).

The second step is to create the post data. To create a new Issue we need to post the following information

• Title
  • field name = title
  • What is it: Title of the issue
• Assigned To:
  • field name = assignedto
  • What is it: The UID of the user to which the new Issue will be assigned. You can find the UID for a user by looking at the Site Users page.
• Area:
  • field name = area
  • What is it: The issue area. See the issues list for list of available areas.
• Type:
  • field name = type
  • What is it: The issue type. See the issues list for list of available types.
• Priority
  • field name = priority
  • What is it: The issue priority. See the issues list for list of available priorities.
• Comment
  • field name = comment
  • What is it: Issue description, repro steps, etc
• Action
  • field name = action
  • value = org.labkey.issue.IssuesController$InsertAction
  • The value is always the same.
• Issue ID
  • field name = issueId
  • value = 0
  • For new issues, the value is always = 0

An example of creating the post data is below

All that is left to do is submit it to the server.

That is all you need to do to programmatically create new Issues on a LabKey Server.

A number of years ago, when GMAIL was just starting to get popular; there was privacy controversy over the fact that Google was scanning the contents of your emails in order to show contextual ads. Just as the tech press was starting to go ballistic over this story, I was making the decision about where to host email for personal domains.

There were two options

1. Build my own email server
2. Use a hosted service. * At this time, given my list of requirements, GMAIL was the only option.

Using GMAIL was and still is the obvious answer. Google runs a mail service so much better than I can. And all the time I would spend installing patches and figuring out why SPAM is getting through, for example, could be spent doing better things like skiing or spending quality time in a hammock in my backyard.

In my head I understood that GMAIL was the way to go, but I could not shake the feeling that running my own server was the best and only option. Looking back I realize the reasons for this were:

1. I wanted to own my email messages. I wanted to make sure I could access messages from yesterday, from 1 year ago and from 15 years ago. They are mine and I do not want a company or government take-down order to block my access to them.
2. (This one is a bit irrational(as email is like a postcard, anyone can read it at any point in the journey)) By hosting my own email server, I could make sure there was one less large corporation reading my email.

As you could guess, I ended up building and running my own email server. It started out at Slicehost and it now resides over at Rackspace Cloud.

With the decision to run a blog at http://www.fourproc.com, I needed to add a few more domains to this email server. In order to do this I had to enable Virtual Hosting support in sendmail. This is surprisingly easy to do.

Add Virtual Hosting support in sendmail

Enable virtusertable feature

Virtusertable is essentially an alias file. It allows you to route incoming email addresses from multiple domains to the appropriate mailbox. (See the virtusertable section at http://www.sendmail.org/m4/features.html for more information)

Edit the sendmail configuration file

vi /etc/mail/sendmail.mc
[added]
    FEATURE(`virtusertable')dnl

Create and configure the virtusertable file. (Below is a sample file)

vi /etc/mail/virtusertable
[added] 
    # Email addresses for fourproc, 4proc
    user1@fourproc.com      brian
    user2@fourproc.com      user2-fourproc
    user3@fourproc.com      user2-fourproc
    user4@fourproc.com      user4-fourproc
    webmaster@fourproc.com  brian
    
    @4proc.com              %1@fourproc.com
    
    # Catch-all addresses for each domain 
    
    @fourproc.com           catchme

Create the virtusertable db file

makemap -r hash virtusertable.db < virtusertable

This table will route incoming messages in the following way

• Incoming messages to user1@fourproc.com will be delivered to brian’s mailbox
• Incoming messages to user2@fourproc.com will be delivered to user2-fourproc mailbox
• Incoming messages to user3@fourproc.com will be delivered to user2-fourproc mailbox
• Incoming messages to user4@fourproc.com will be delivered to user4-fourproc mailbox
• Incoming messages to webmaster@fourproc.com will be delivered to brian’s mailbox
• Incoming messages to any other email address in fourproc.com domain will be delivered to the catchme mailbox
• Incoming messages to any email address in 4proc.com domain will be delivered corresponding user in the fourproc.com domain.

Enable generic_entire_domain feature

The genericstable and generics-domain features control the routing of outgoing email address. If the genericstables and the GENERICS_DOMAIN_FILE features are enabled, then sendmail will masquerade the from address and the envelope if the FROM addresses domain is in the GENERICS_DOMAIN_FILE file. (See the genericstable section at http://www.sendmail.org/m4/features.html

Edit the sendmail configuration file

vi /etc/mail/sendmail.mc
    [added]
    FEATURE(`genericstable')dnl
    GENERICS_DOMAIN_FILE(`/etc/mail/generics-domains')dnl

Create and configure the genericstable. (Below is a sample file)

touch /etc/mail/genericstable 

For my case, no entries in the table were needed. Create the generics table db file

makemap -r hash genericstable.db < genericstable

Create and configure the generics-domains file. (Below is a sample file)

vi /etc/mail/generics-domains
[added]
    domain1.com
    fourproc.com
    4proc.com
    domain2.com
    domain3.com

Start sendmail using the new configuration

The first thing I need to do is create the new sendmail.cf configuration file

cd /etc/mail
make -C /etc/mail

Now I can restart the sendmail server

/etc/init.d/sendmail restart 

That was it. It was pretty simple. Took me a few hours total to research how to do it, test the configurations on a spare AWS instance and rollout to the production server. This was probably 8X longer than it would have taken me to rollout these domains using Google Apps. Maybe sometime in the future, my reasons for running my own email server will no longer matter to me and I will move over to GMAIL or another service. But for now, I am sticking with running my own.

Today, I moved this blog from Posterous to the open web (ie a cloud server I host at RackSpace). This blog is just a bunch of static pages, images and css. No corporate blogging silo. No database. Many fewer moving parts.

The blog is generated using jekyll and Bootstrap. I will be posting my implementation notes for installation, configuration and content generation for this blog in a few days.

The blog is a bit barebones right now. Comments, RSS feed and tag pages will be available shortly.

I hope you find it useful.

-Brian

PS: I know I that I still use the Posterous for my tripReports blog. I am getting to it, give me a little time.

PSS: I like Posterous and it has served up my sites very nicely over the years. But I want to make sure I own my content and always have access to it, no matter if the company suddenly does not like me or the governments decides to put the company under a takedown order. (I still have not figured out how to make a copy of my original trip reports blog. I made it so secure, I cannot get into and download all the content hehehe…. )

* Update: Modified and slightly revised version of this post is available on the LabKey Blog in two parts. See Part1 and Part2

As you may or may not know, the LabKey Server has a whole suite of client APIs that can be used to programmatically access data inside your LabKey Server. Like any other open source project, some of these APIs have seen a lot more development than others (see the functionality available in the Javascript API, which is one of the first we developed; compared to our newest API, Python which was written last month by Elizabeth in her spare time).

Periodically, I will need to access data in a LabKey Server in an unsupported language(for example NODE.js or BASH). In these cases, one of the first hurdles that you need to get over is simply authenticating to the server. In this entry, I am going to provide some sample code for how to authenticate to the LabKey Server in a variety of languages (shell script, python and node.js).

How Does Authentication in the LabKey Server work?

LabKey Server uses form-based authentication, by default, for all user-agents (ie browsers). However, it will correctly accept HTTP basic authentication headers if presented. I recommend using Basic Auth when programmatically accessing the server.

Please note that when using either form-based authentication or basic authentication, your credentials are passed in clear text, thus I strongly recommend implementing SSL on your LabKey Server.

Here is a very simplistic example of how Basic Auth works:

1. You attempt to connect to a web page which requires authentication
2. The server will respond with an Authorization Required error message (HTTP response code = 401).
3. In addition, the server will add a new HTTP header to the response. It will be similar toWWW-Authenticate: Basic realm="Default"
4. If you are using a browser, the browser will then pop-up a dialog box asking you to enter a username and password to gain access.

This is how the LabKey Server works for all the APIs, which are designed to be accessed programmatically.

For all other pages(where the LabKey Server assumes you will accessing via a browser) the LabKey Server will not behave in the way described above but will behave completely different. Here is an example:

1. You attempt to connect to a web page which requires authentication
2. The server will respond with an Moved Temporarily error message (HTTP response code = 302).
   • The server does not include the WWW-Authenticate:... header in the response headers
3. If you are using a browser, the browser be redirected to the LabKey Server login page.
4. If you are not using a browser, but accessing the server programmatically you simply get this error message

In general, the actions which respond with the WWW-Authenticate:... HTTP Header are

• All API requests (any page whose names ends with .api)
  • Both POST and GET

All other pages do not respond with the WWW-Authenticate:... HTTP Header, but redirect you to the LabKey Server login page. This includes actions such as

• Most Admin actions (creating new folders, viewing the Audit Log)
• Submitting MS2 searches to the server
• Viewing and managing Wikis
• Viewing or posting messages to a Message board or Issue tracker.

This is an important point. When accessing the LabKey Server programmatically tools (wget, curl, etc) or programming libraries which handle Basic Auth authentication will work properly when accessing an API, but will not work when accessing any other page on the server.

A very easy way to determine the authentication behavior of page is to make an unauthenticated connection to the web page and review the response HTTP Headers. You can do this using python

or using wget

wget --server-response http://my.labkey.server/labkey/your/page

Let me use an example to illustrate the behavior I explained above. Lets use wget to view a secured wiki page on the server

wget --server-response --max-redirects=0 https://www.labkey.org/wiki/Home/page.view?name=secureWiki

the response is

HTTP request sent, awaiting response... 
  HTTP/1.1 302 Moved Temporarily
  Server: Apache-Coyote/1.1
  Expires: Thu, 01 Jan 1970 00:00:00 GMT
  Cache-Control: no-cache
  Location: /login/Internal/login.view?returnUrl=%wiki%2FHome%page.view%3Fname=secureWiki
  Content-Length: 156
  Date: Wed, 24 Aug 2011 01:01:19 GMT
  Connection: keep-alive
Location: /login/Internal/login.view?returnUrl=%wiki%2FHome%page.view%3Fname=secureWiki [following]

You can see that server:

1. Does not respond with a WWW-Authenticate:... HTTP Header, but
2. Redirects the browser to the login page

Now lets look at an attempt to use the getQuery API (which you can use to query data in the LabKey Server).

wget --server-response --max-redirects=0 https://www.labkey.org/query/Internal/getQuery.api

the response is

HTTP request sent, awaiting response... 
  HTTP/1.1 401 Unauthorized
  Server: Apache-Coyote/1.1
  WWW-Authenticate: Basic realm="LabKey Server and CPAS Distribution and Support"
  Content-Type: text/html
  Content-Length: 2871
  Vary: Accept-Encoding
  Date: Wed, 24 Aug 2011 01:05:35 GMT
  Connection: keep-alive

In this example, you can see that the server

1. Responded with a HTTP response code of 401
2. Added the WWW-Authenticate: Basic realm="LabKey Server and CPAS Distribution and Support" header which you can then programmatically use to authenticate using Basic Auth.

Now that I have shown you how the LabKey Server will respond to authentication requests, I can provide some examples of how to authenticate to the LabKey Server using various programming languages. Before we get to the examples, here are few links to documentation for programmatically accessing a LabKey Server and using the LabKey APIs

• Working with the HTTP Interface and APIs
• Examples of using APIs
• General Developer Documentation

In the examples below, you will see two URLs used; one is meant to represent viewing a secured wiki page and the other is for reading the contents of a list. The URLs are:

• Reading a secured wiki: https://your.labkey.server/labkey/wiki/Home/page.view?name=securedWiki
  • To use a wiki from your server, simply find a secured wiki on your server, copy the URL and paste it into the example code.
• Reading the contents of list: https://your.labkey.server/labkey/query/Home/getQuery.api?schemaName=lists&am...
  • This URL will read the contents of list named listName in the Home project.
  • Simply replace Home with the container path for your list and listName with the name of your list.

Authenticating to LabKey Server using Python

LabKey currently ships a client PYTHON client API. The PYTHON client api currently supports (as of 01/01/2012):

• reading data inside LabKey Server (selectRows)
• inserting new data inside LabKey Server (insertRows)
• deleting existing data inside LabKey Server (deleteRows)
• updating data inside LabKey Server (updateRows)
• executing SQL statements (executeSql)
• updating a wiki page
• posting a message to a message board

The client API will handle the authentication for you. If there are other tasks you need to perform, you can use the example code below to authenticate to the server

Authenticate to an API page

In this example, I will use the urllib2 library. I will load my creadentials into the a password manager and then let the OpenerDirector object handle passing the Basic Auth headers to the server when requested.

Authenticate to an non API page

In this example, I will load my creadentials into the a password manager and in addition add the Basic Auth Header to all requests using the OpenerDirector object.

NOTE: If you want to use only one opener for both API and non-API pages, you simply can use the 2nd example for accessing all pages.

Authenicating to LabKey Server with Node.js

There is currently not a client API for the Node.js language yet. (If you are using Node.js to work with data in your LabKey Server and want to assist in writing an client API, please post a message to the Developer Community Forum as we would love to work with you.)

As we saw above, with Python, there are two ways to authenticate. For API pages, the OpenerDirector object handles chaining of requests together and will properly resubmit a request if it receives a 401 HTTP response code with theWWW Authenicate: header. There is not something similar for NODE.js and thus there is only one way to authenticate to the server.

NOTE: Full disclosure, I am a Node.js novice, there are probably many better ways to do this. If you know a much better, simpler way, please post it in the comments and I will update this example.

Authenticating to the LabKey Server using wget

Now, if you do not want to bother with these programming languages and just want to grab a page quickly or download some data and parse it with sed, you can just use wget. (To be completely honest, I use wget all the time for testing and/or grabbing a single page).

Authenticating to an API page.

This case is pretty simple one liner

wget --server-response --http-user=myEmail --http-password=myPassword \
 "https://your.labkey.server/labkey/query/Home/getQuery.api?schemaName=lists&query.queryName=listName"

Authenticating to a non-API page.

For this case, you will need to make two wget requests. The first to login and the 2nd to access the secured page.

wget --server-response --save-cookies cookies.cpas --keep-session-cookies \
https://your.labkey.server/Login/login.post --post-data "email=myEmail&password=myPassword"

wget --load-cookies cookies.cpas https://your.labkey.server/labkey/wiki/Home/page.view?name=securedWiki

Now that you have gotten past the authentication, you can start doing all sorts of fun things with your data.

There are a few other languages that I have used in the past, namely perl, that I should have included in these examples. I will try and get to those in a later post(s). (If you are using perl, see the Perl Client API for an example)

I hope this is helpful and understandable. If you have any questions, please post a comment or post a message to the Developer Community Forum