Ubuntu Server

Python 3 + Oracle on Ubuntu Server 14.04

Posted on by Matt Woodward

I’ll invite my readers to check my previous post on this topic to get a sense of how I feel about Oracle.

Small updates it seems to get cx_Oracle working with Python 3 on Ubuntu 14.04. Assume sudo/root for all the following.

  1. Download the RPM versions of both the basic and SDK clients for 11.2.0.4 from http://www.oracle.com/technetwork/topics/linuxx86-64soft-092277.html
    1. Note: If you’re not familiar with the idiotic hell that is Oracle downloads, you have to do this on a browser where you can click a radio button and THEN download, so wget from the server on wish you actually want to install this stuff is unfortunately a no go.
    2. The 12.x version of this stuff still doesn’t work with cx_Oracle. At least not for me.
  2. scp the rpm files to the target server
  3. ssh to the target server
  4. apt-get install libaio1 alien
  5. For both the RPMs:
    alien -d RPM_FILE_NAME
  6. For both the newly created Debian packages:
    dpkg -i DEBIAN_PACKAGE_NAME
  7. touch /etc/ld.so.conf.d/oracle.conf
  8. echo “/usr/lib/oracle/11.2/client64/lib” > /etc/ld.so.conf.d/oracle.conf
  9. echo “export ORACLE_HOME=/usr/lib/oracle/11.2/client64” >> /etc/environment
  10. echo “export LD_LIBRARY_PATH=$ORACLE_HOME/lib” >> /etc/environment
  11. source /etc/environment
  12. ldconfig
  13. pip3 install cx-oracle
  14. Put the relevant tnsnames.ora in $ORACLE_HOME/network/admin
Note that unlike my previous instructions you can install cx_Oracle using pip3 once you have everything in place. Maybe you could use pip before but I seemed to have better luck with the converted RPM, and at that point I was kind of tearing my hair out so I probably didn’t go back to take a run at it using pip once the Oracle bits were sorted.
The big change here is the necessity to install both the basic client and the SDK client packages for Oracle; it worked with Python 2.7 on Ubuntu 12.04 without the SDK package.

Python + Oracle on Ubuntu Server 12.04

Posted on by Matt Woodward

Affectionately known among all non-masochists in the world of IT as The Seventh Circle of Hell (with real hell being preferable), working with Oracle is always a hair-tearing nightmarish fork-in-the-eye please-for-the-love-of-god-kill-me-now experience that none but those who look to Ted Bundy, Jeffrey Dahmer, and John Wayne Gacy for moral and spiritual guidance would wish upon even their most reviled enemies.

Yes, it’s that bad. And apparently nowhere is it worse than when one attempts to get Oracle working with Python on Ubuntu.

I’m not even talking about installing the Oracle database server itself here people, I’m just needing a Python application to talk to an existing Oracle database. One would think, as with every other database server on the planet (and yes, I’m including that other slice of hell SQL Server in that statement since it’s a damn sight simpler to get working — even on Linux — than Oracle), you’d simply apt-get and/or pip install a library or two and be done with it.

If you actually do think that, you’ve already forgotten that this is Oracle we’re talking about.

That said, one does what one has to do to keep the paychecks coming, so if you need to do this here’s the steps to make it all happen. (Note that on Step 1 I’m assuming you have already installed all the other Python packages you may need. I’m focusing on the stuff you may not have that you definitely need.)

  1. sudo apt-get install libaio1 alien
  2. Download the RPM of version 11.2.0.4.0 of the Oracle client from http://www.oracle.com/technetwork/topics/linuxx86-64soft-092277.html (note that as of the date of this writing the 12.x version doesn’t work, or at least didn’t for me)
    1. You have to have an Oracle Web Account and use that to log in and download this, which makes using wget on the target server itself or automating the process for use with something like a Vagrant provisioning script rather problematic. Short version, you’ll have to download this locally and then scp it up to the target server. What I did is downloaded and converted this RPM as well as the other necessary RPM, converted them once, and put them in a git repo from which I can clone in my Vagrant provisioning script. Whether or not that adheres to the licensing agreement, I don’t know and I don’t care. If you’re paranoid, check with a lawyer before repeating my solution on this.
    2. If the version number differs slightly from what I have here, adjust later steps accordingly.
  3. Download the RPM of the Python 2.7/Oracle 11g version of the cx_Oracle Python libraries from http://cx-oracle.sourceforge.net/
  4. scp the RPMs up to the target server as needed.
  5. Convert the RPMs to Debian packages using alien:
    sudo alien -d FILENAME.rpm (where FILENAME is of course the name of each of the two RPM files)
  6. sudo dpkg -i oracle-installclient11.2-basic_11.2.0.4.0-2_amd64.deb
  7. sudo vim /etc/ld.so.conf.d/oracle.conf
    1. Note: this file won’t already exist, so you’ll be creating this as a new file in this step
  8. Enter the following in the newly created oracle.conf file and save it:
    /usr/lib/oracle/11.2/client64/lib
  9. export ORACLE_HOME=/usr/lib/oracle/11.2/client64
  10. export LD_LIBRARY_PATH=$ORACLE_HOME/lib
  11. sudo ldconfig
  12. sudo dpkg -i cx-oracle_5.1.2-2_amd64.deb
  13. cd /usr/lib/python2.7
  14. sudo mv site-packages/cx_Oracle* dist-packages
  15. sudo rmdir site-packages
  16. sudo ln -s dist-packages site-packages
  17. Verify installation by opening a Python interpreter and run the following:
    import cx_Oracle
    1. If you don’t get an import error, everything is working properly

As far as automating this for use with Vagrant,  in my provisioning script I simply echoed the export statements in the steps above into /etc/environment, did source /etc/environment and followed that with ldconfig. Other than that the steps in the bash script are pretty much what’s above, but if people are interested in seeing the script let me know and I can post it.

And there you have it. A lot of trial and error and head bashing went into that final solution, and since I kind of cobbled together the steps from various resources I’ll post those below in case you want to see some of the other solutions and source material.

Happy Oracleing. Or not.

References

  1. http://iambusychangingtheworld.blogspot.com/2013/06/python-oracle-sqlalchemy-on-ubuntu-1304.html 
  2. http://maxolasersquad.blogspot.com/2011/04/cxoracle-on-ubuntu-1104-natty.html 
  3. https://linuxindetails.wordpress.com/2009/12/26/installation-of-python-cx_oracle-module-for-debian-squeeze/
  4. http://stackoverflow.com/questions/12538238/python-module-cx-oracle-module-could-not-be-found

CouchDB Tip: When You Can’t Stop the Admin Party

Posted on by Matt Woodward

I was setting up a new CouchDB 1.2 server today on Ubuntu Server, specifically following this excellent guide since sudo apt-get install couchdb still gets you CouchDB 0.10. Serious WTF on the fact that the apt installation method is years out of date — maybe I should figure out who to talk to about it and volunteer to maintain the packages if it’s just a matter of people not having time.

The installation went fine until I attempted to turn off the admin party, at which point after I submitted the form containing the initial admin user’s name and password things just spun indefinitely. And apparently adding the admin user info manually to the [admin] section of the local.ini file no longer works, since it doesn’t automatically encrypt the password you type into the file on a server restart like it used to.

Long and short of it is if you see this happening, chances are there’s a permission problem with your config files, which are stored (if you compile from source) in /usr/local/etc/couchdb. In my case that directory and the files therein were owned by root and I’m not running CouchDB as root, so when I tried to fix the admin party the user that’s running CouchDB didn’t have permission to write to the files.

A quick chown on that directory structure and you’re back to being an admin party pooper.

Setting Default umask for SFTP on Ubuntu Server

Posted on by Matt Woodward

Much as described in this blog post by Jeff Robbins, I have a situation where two sftp users in the same group are both uploading files to an Ubuntu 10.04 server using Dreamweaver. The issue is that by default the permissions are 755, so even though both users are in the same group, only the file owner has write permissions. Since the users need to be able to overwrite each other’s files I needed a way to have the default permissions be 775.

What is outlined in the blog post above is exactly what I was after, but for some reason on Ubuntu server if you use what is the final edit in that post:
Subsystem sftp /usr/lib/openssh/sftp-server -u 0002
That results in “Connection closed” messages when you try to log in. The solution above that one works, just note the minor modification of pointing to /usr/lib/… instead of /usr/libexec/…
Subsystem sftp /bin/sh -c ‘umask 0002; /usr/lib/openssh/sftp-server’
Restart ssh and you should be in business.
Thanks to Jeff for that very helpful blog post, and to Thad Meyer for pointing it out to me just last week (coincidentally enough).