Microscopy in Multiples

The microscope has come to symbolize the pursuit of scientific research. Today, disciplines ranging from biology to industrial metrology rely on a variety of high-resolution optical imaging technologies such as brightfield, darkfield, differential interference contrast (DIC), and fluorescence microscopies. Almost all of them can benefit from faster operation. Up until now, progress was hampered by a fundamental limitation of single-optical-axis imaging, namely the tradeoff between field of view (FOV) and image resolution. For most optical systems, this tradeoff means that either we can image a large area at low resolution or we can obtain a high-resolution image of a small area.

An array approach offers a way around this limitation using a large number of miniature microscopes to image large areas at high resolution. Thanks to the benefits of parallel image acquisition and parallel processing, such a system can scan images at a much higher speed than conventional systems and process digital data in real time.

The tradeoff between FOV and resolution becomes acutely apparent in medical microscopy systems, with which there is often a need to image relatively large areas of a microscope slide at high resolution. State-of-the-art field-flattened objectives deliver FOVs of only about 1 mm in diameter. To image objects spanning more than one FOV of such an objective, researchers have developed three approaches: tiled imaging, push-broom scanning, and flying-spot (whisk-broom) scanning. The tiled approach is based on acquiring a number of 2-D images (tiles) with some degree of overlap and stitching them together into a composite image using an appropriate algorithm. The sample stage must stop after acquisition of each image tile; thus, in addition to being computationally involved, tiling is susceptible to the mechanical inaccuracies of the scanning stage.

The push-broom scanning approach, which combines a linear detector with the continuous motion of the stage, provides an alternative. The image is acquired in the form of a swath of arbitrary length and a width equal to the FOV of the objective. This approach requires the stitching of swaths rather than tiles and is subject to errors introduced by non-repeatable stage performance.

The flying-spot approach, which requires a single detector, images with a single illumination spot that sweeps the object in the field of view. Depending on implementation, the flying-spot approach shares problems of either tiled or push-broom scanning.

All three methods require significant time to complete the imaging operation and extensive computation to construct the composite high-resolution image. As a consequence, there is a delay of minutes to hours between image acquisition and viewing, unacceptable for many applications. Solving the FOV/resolution challenge requires a new approach.

One solution is to use several microscopic systems to simultaneously image different portions of the object. Typical microscopic objectives are characterized by an FOV-to-physical-lens-diameter ratio (FDR) on the order of 25 to 50, however. Aggregating conventional objectives into assemblies to simultaneously image small objects is not practical - the images acquired from each objective would be spaced too far apart from each other on the object.

The solution comes from the field of micro-optics. Our optical engineers and researchers from the University of Arizona's Optical Sciences Center (Tucson, AZ) have developed a new generation of optical systems with FDR values around eight. These optical designs achieve low FDR values by relying almost exclusively on aspheric surfaces, thus minimizing the number of lens elements per objective. The small dimensions and reduced FDR of the lenses allow us to assemble a large ensemble of miniature microscope objectives to simultaneously image different parts of the same object at high resolution.

Figure 1. The 8 X 10 array of miniature microscope objectives is constructed from several monolithic plates. Precise alignment of the plates is critical to the performance of all objectives in the array.

The system consists of a 2-D array of multi-element micro-optical systems overlaying a 2-D detector array (see figure 1). At any instant, the array captures a complex, disjointed frame of image data. To form the complete image of an extended area on the microscope slide, for instance, the array microscope requires only a one-directional scan along the length of the slide (see figure 2). The approach is akin to push-broom scanning in that it uses a linear detector and the motion of the sample stage. Each objective in the microlens array magnifies the object, so it is not possible to image the entire sample with 2-D detectors at once—the images of the adjacent sections of the object would simply overlap.

Figure 2. The array of microscopes requires a one-directional scan of the object to form a complete, large-format, high-resolution image.

Our solution spaces the optics such that the magnified image does not overlap on the detector and requires that "sampling" of the object does not cover it in a contiguous way. We use an eight-row linear detector array, each row containing 10 optical systems (as explained below) to achieve full coverage of the object. It is simpler to design and fabricate linear detectors than 2-D arrays: There is more space per pixel on the die, and that space can be used for additional circuitry to accomplish correlated double sampling, for example.

Such an optics/image-sensor configuration produces image data in the form of 80 swaths as the sample stage advances the object. The array objective and the image sensor form one rigid assembly and allow us to calibrate it in such a way that we know where each swath should start and end to form a seamless image. Programmable electronics extract from the image formed by each objective just the segment that will later fit with its neighbors seamlessly. Because the image sensor and array of objectives are mechanically stable and the width of the assembly spans the entire sample, we do not need to perform stitching. We simply rearrange the swaths to form the final image. The only requirement that we pose on the sample stage is that it needs to move straight and flat. We have solved this problem by building our own stages on very flat glass blocks.

Each of the 80 individual optical systems has a numerical aperture of 0.65 and an FOV on the order of 250 µm in diameter. Each objective is designed primarily using aspheric surfaces, with clear apertures not exceeding 2 mm in diameter. The objectives are laid out on an 18 mm ¥ 10.5 mm grid; the array is comparable in height to a stack of four coins with an object-to-image distance (throw) of less than 10 mm.

One challenge in the realization of array microscopy is the fabrication of large numbers of identical lenses with characteristics that are sufficiently well matched to allow simultaneous imaging of the object at the same conjugates and without variation of imaging performance across the array. The difficulties range from analysis of tolerances (typical tolerancing tools are designed to evaluate a single imaging system, not an ensemble of systems), to fabrication of aspheric optics with sufficiently high accuracy and precision. The lenses in the array are located so close together that assembling them from individual components is possible in principle but not practical; thus, they must be produced in arrays by methods such as injection molding, lithography, or direct laser writing and are limited to materials that can be either molded or embossed.

The detector presents another challenge. To achieve the desired FOV and resolution, the lens array requires a large-format, small-pixel image sensor. Smaller pixels allow optics to work at smaller magnification while yielding denser packing for a smaller overall package. Our magnification factor of seven gives us about 0.5 µm resolution at the object.

Unfortunately, most commercial sensors exhibit a correlation between physical size and pixel size. Our solution was to design a custom complementary metal oxide semiconductor image detector to match the requirement for pixels small enough to match the low magnification of optics that need high-speed parallel data acquisition for array microscopy. The resultant image sensor captures data at 2500 frames/s with its 3.3-µm pixels. It has 10 parallel output channels and operates at a master clock rate of 15 MHz; the current-generation array microscope produces image data at a rate of 120 MB/s. We diminished noise with true correlated double sampling. Custom hardware compresses the acquired image data in real time to reduce the data export rate to the host PC.

A final challenge is autofocusing. An automated system for imaging histopathology slides, for instance, needs to locate tissue and make intelligent choices about the best focus plane for a particular section of a microscope slide. In some cases, the preparation of pathology slides leaves a layer of uneven mounting medium or bubbles of air embedded in the sample. These problems present difficulties for any automated scanning system, but the challenge is multiplied in array microscopy by the large number of objectives that need to be optimally focused on the object simultaneously.

To solve this problem, we suspended the optical array with the detector and relevant electronics on a scanner platform with the ability to change not only the optic-to-sample distance (z-direction), as is the case with conventional microscopes, but also pitch and roll. These added degrees of freedom allow the system to determine the optimum imaging position for each section of the scanned object and create a trajectory for the array-microscope optics to move across the microscope slide while images are collected. We accomplish this articulation with three custom precision flexure stages whose positions are continually adjusted during the movement of the sample slide, controlled by an optical feedback loop. This solution proves very effective even when imaging is difficult, for example with non-flat tissue samples and our array microscope.

Even with the first-generation image sensor operating at a modest 15-MHz clock rate, our array microscope proved to be five to 10 times faster than conventional systems. The design of the array as a monolithic ensemble of objectives does not require any form of stitching or image post-processing, making the completely reconstructed image available immediately after completion of a scan. We based the supporting electronics on programmable logic and offer the additional advantage of on-the-fly computing and image-processing operations such as edge detection, convolution, or image histogram computation. The results of such analyses are also available instantaneously at the end of scanning.

Such capabilities have the potential to impact fields such as life sciences and healthcare. The current version of the array-microscope scanner is capable of imaging 20 to 40 slides per hour at 54,000 dpi, enabling it to keep up with the work load of an average histopathology laboratory while allowing 100% digitization of the slides.1 Such a shift to digitization is expected to transform pathology in the same way that digital imaging has transformed the practice of radiology.

The array approach can be extended to other approaches such as epi-illumination microscopy, multispectral micro-scopy, DIC, or epi-fluorescence. We anticipate that the development of ultra-fast, high-resolution microscope systems will launch the next chapter in digital microscopy, namely the development of progressively more sophisticated analysis methods to extract information from image data. oe

 References

1. R. Weinstein et al., Human Pathology, 35[11], p. 1303 (2004).