Design Document for Image Depositor

Introduction

Currently, it takes much effort to deposit a scan image into the database. The user has to first convert the image, which is in tld format, into mrc format. then go to the browser, login to the database web site, fill the form for a single scan image with all the scanner parameters and scan image. then click 'submit' button to add this image into the database.

In addition, it is very slow to download an image file from the database server web page.

This project is to develop a program to automate this process.

Reference

• Requirements.

Unresolved Issues

Design

Because the database is in Zope server, we implement the program in Python, a good script language to talk with Zope. Also, use python module xmlrpclib.py shipped together with Zope to send the scan image and scanner parameters to the database.
1. This program should be able to convert a tld format image into a mrc format image.

   Call Steve's tld2mrc program to do this. If the file is already a mrc image , no conversion need to be done. The file type is determined by its extension.

2. Before deposited into the database, the generated mrc format image should be hided from the user.

   Store the mrc format image in /var/tmp/ directory with a tempory file name.

3. The image should be deposited together with all the scanner parameters.

   This is the hardest part of this project. It can be divided into two steps:

   1. Find the micrograph location given its film id. This is the place Scan object is deposited.

      Use automatic cataloging to do this. Zope uses ZCatalog objects to search the database. If the searched objects are defined as "Catalog-aware", then they can be automatically cataloged. This means that when objects are created, changed, and destroyed, they are automatically tracked by a ZCatalog object. The ZCatalog object must be named "Catalog" and be put under the parent folder of all tracked objects.

      For this project, do the following:

      1. Create a ZCatalog object named "Catalog" under 'EMDB' folder. It has a field index named "filmid", and a meta type named "absolute_url".
      2. Write a python script to search the catalog. Its input is the filmid. Its output is the micrograph's absolute URL. If nothing is found, return a null string. If multiple entries are found, return all of them, and let the user to choose.

   2. Deposit the scanner parameters and image.

      The solution is to emulate the current web-based implementation.

      Let me first explain the current web-based implemention. The user first fills out the scan image form with all the scanner parameters and upload the image file from local directory. Then he/she clicks submit button. the data will be deposited into the database. What happens inside the server is that when the user clicks 'submit' button, a method called "Scan_add" will be called. The input to this method is an object called "REQUEST", which stores all the form data. "Scan_add" method builds a Scan object based on "REQUEST".

      For this program, we are going to pass the parameters from user side to the server side through xmlrpc. In the server side, use a method, similar to "Scan_add", to set up the REQUEST object manually using these parameters. Then use REQUEST to construct a "Scan" object. All the scanner parameters can be passed to the server using this way.

      For the image, first ftp the image to the database local computer. Then pass the local file path to the database server. A program in the database server will upload the image into the database as an external image. After the image is uploaded, the original copy in the local file system is removed by the server. The ftp account is upload-only. The user cannot delete any file from it.

4. The program should allow a user to deposit one or more tld format images in a single run.

   use a loop to go through each image file. First collect all the image parameters, which requires the user's input. Then do the conversion and deposit, which takes longer time.

5. The user needs a username + password to run this program.

   use python's getpass module to get user's username and password. Then use xmlrpclib's BasicAuthTransport class to send an authorized header to the Zope server.

---

lpeng@bcm.tmc.edu
Last modified: Thu Jan 10 14:12:27 CST 2002