The Design and Engineering of Curiosity

by Emily Lakdawalla

  Figure 7.7. How hot pixels, shutter smear, and JPEG compression can reduce Mastcam image quality. Taken as part of a drive direction panorama, this Mastcam image was returned to Earth with fairly high JPEG compression (quality 65). Mastcam image 1030ML0045010040305530E01. NASA/JPL-Caltech/MSSS/Emily Lakdawalla.

  Images containing particularly bright objects – such as parts of the rover, or hot pixels – can be affected by shutter smear. The Mastcams have no physical shutters. Instead, once the camera has exposed the detector for the requested length of time, it shifts the charge out of the exposed area and into a shielded area called the transfer cell, one line at a time. All the rest of the lines in the image are shifted upward as each line is moved into the transfer cell and read out. While this is all happening, the detector is still being exposed to the scene. As long as the readout of the image happens quickly relative to the exposure time, it’s hard to notice the effects of shutter smear. But if the scene is bright enough that the exposure time is short – or if the scene contains a very bright pixel – there may be a vertical bright smear streaking the image, running down from bright pixels. The hot pixel that appeared on the left Mastcam on sol 834 is bright enough to create such a streak on all left Mastcam images acquired since it appeared (Figure 7.7). Fortunately, the Mastcam CCDs are large enough that, although cosmetically annoying, the blemishes are not harmful to science.

  The JPEG compression means that most images have some compression artifacts. JPEG compression works on 8-by-8-pixel blocks, and the boundaries of those blocks are often visible. JPEG compression is more effective in places with smooth variations in color and brightness, but can introduce strange artifacts in areas of high contrast. Areas where there is high contrast or a lot of variation also tend to be areas of scientific interest – for instance, in an area of very finely laminated rock layers in alternate Sun and shadow (Figure 7.7, bottom). Where the compression artifacts make it difficult to interpret the geology, the team can choose to re-transmit the image with less compression, or even losslessly – as long as the original image is still stored in the camera’s flash memory. As of early 2017, not quite half of all of the Mastcam data had been returned a second time with lossless compression. At that time, the mission switched to returning all non-time-critical Mastcam science data losslessly, accepting delayed data return in exchange for a larger proportion of losslessly compressed data and a reduction in the complexity of data curation.6

  7.2.1.6 Calibration target

  The Mastcam calibration target is a flight spare of the ones flown on the Mars Exploration Rovers whose design was modified by the addition of magnets. It is mounted on the rover deck, 1.2 meters away from the cameras, on top of the box that houses the rover pyro fire assembly (Figure 7.8).

  Unfortunately, because of the location of the hardstop on the azimuth actuator of the remote sensing mast, it is not possible to image the calibration target through both Mastcam eyes with a single pointing. The mast head has to rotate almost 360° in order to image the calibration target through both Mastcams. (This problem will be solved for Curiosity’s successor, Mars 2020, by moving the calibration target just a few centimeters toward the center line of the rover.) The right Mastcam is somewhat farsighted, with a minimum in-focus range of 1.6 meters, so its images of the calibration target are not in focus, but that doesn’t affect the calibration target’s usefulness.

  Like the Earth sundials that the calibration target resembles, the Mastcam calibration target has art and text embellishments. Most of these are inherited from the Mars Exploration Rover Pancam calibration target, but there is a new motto: “Mars 2012” at the top of the dial (the year is usually hidden from view behind the gnomon as seen by Mastcam, but it’s visible to MAHLI), and “to Mars to explore” at the bottom. On the four vertical sides the following text is engraved:For millennia, Mars has stimulated our imaginations. First we saw Mars as a wandering red star, a bringer of war from the abode of the gods.

  In recent centuries, the planet’s changing appearance in telescopes caused us to think that Mars had a climate like the Earth’s.

  Our first space age views revealed only a cratered, Moon-like world, but later missions showed that Mars once had abundant liquid water.

  Through it all, we have wondered: Has there been life on Mars? To those taking the next steps to find out, we wish a safe journey and the joy of discovery.

  The calibration target is 8 centimeters square and has 7 regions useful for calibration, including 4 color chips at the corners and 3 grayscale rings around the black gnomon (central post). Ring-shaped “sweep” magnets underneath the color chips and the lighter two of the grayscale rings attract Martian dust to them, keeping the centers of the magnets less dusty than the rest of the calibration target.7 The top of the rover pyro fire assembly has, unfortunately, turned out to be one of the dustiest spots on the rover. The Mastcams image the calibration target whenever they do multispectral imaging, using the same set of filters as were used for the science observation. Although the dust is obscuring the areas intended to be used for calibration, the dust affects the brightness and color of the calibration target in a way that is straightforward to model, so it remains a useful calibration tool.

  Figure 7.8. Mastcam calibration target as seen over time. NASA/JPL-Caltech/MSSS/Emily Lakdawalla.

  7.2.2 Using Mastcam

  As with the Navcams, Mastcam imaging can either be targeted, or “blind.” To do targeted imaging, the Mastcam team needs Navcam images to provide spatial information. Targeted imaging doesn’t just happen at drive stops: Curiosity can perform targeted Mastcam imaging in the middle of a drive as long as the drive has not taken Curiosity beyond the terrain mesh calculated at the last drive stop. Blind imaging doesn’t require Navcam context. Blind observations include 360° panoramas, Sun and horizon observations (since the position of the Sun and of distant landscape features don’t measurably change over the course of a short drive), and observations in rover-relative locations, like the arm work volume and ChemCam targetable region.

  7.2.2.1 360° panoramas

  The rover acquires 360° panoramas with the wider-angle left Mastcam in interesting science locations and also roughly every 250 to 300 meters along long traverses in order to document the landscape. Full panoramas need no specific targeting (except coarsely, to include the top of Mount Sharp), so they are often taken on restricted sols when the rover’s precise state isn’t known. It takes 23 left Mastcam frames to complete a single tier over the full azimuth range of 360°, at about 8 minutes per tier, and roughly an hour for a typical complete panorama. One tier is centered near the horizon; each subsequent tier drops by 12°; and a partial tier covers the predicted location of the peak of Mount Sharp.

  To save on time and bandwidth, the tiers below the horizon are usually incomplete, with areas including the rover deck skipped. This is a source of frustration to mission team members and the public alike, who would like to see a Mastcam self-portrait of the rover on the Martian surface including the robotic arm (which can’t be imaged in MAHLI self-portraits). To date, only one Mastcam panorama has included all of the rover deck; it was taken on sol 1197, at Namib dune.

  The higher-resolution right Mastcam requires 9 times as many images to cover the same amount of terrain as the left Mastcam, so has only taken a 360° terrain panorama once, in imaging sessions conducted from sols 172 to 198, while at John Klein.

  7.2.2.2 Tactical support imaging

  The left Mastcams are often used to take 5-by-1 “drive direction” mosaics to survey distant terrain in more detail. Unlike Navcam drive direction panoramas (section 6.​4.​2), the Mastcam panoramas don’t include right-eye imaging, so do not contain depth information of tactical value. They can be shot blind as long as the rover drivers have a good estimate of the direction they will want to travel. They are useful for helping rover drivers avoid rocks that could damage the wheels. Other tactical planning products include mosaics of the work volume in front of the rover to prepare
for in-situ work (see Figure 3.​12), and mosaics of the region targetable by the ChemCam laser in front of and to the right side of the rover, to improve ChemCam target selection. The Mastcams are also used to inspect the turret before and after drilling and sampling operations, to image instrument inlet covers before and after sample delivery, and occasionally to capture movies of sample handling events. They are used to document the health of the wheels, but due to the position of the mast on the rover’s right shoulder, the Mastcams can only see the wheels on the right side of the rover.

  7.2.2.3 Mastcam science imaging

  Most science imaging is done through the clear filter (producing RGB color images, with infrared light cut off) unless otherwise noted.

  Science images and mosaics. Both Mastcams are used to obtain targeted observations of areas of scientific interest. The images may be used for science in and of themselves for study of geomorphology, or may provide valuable context to other types of science data. If a region of interest is larger than a camera’s field of view, the Mastcam team will sequence a mosaic made of slightly overlapping images that can be assembled on Earth later.

  Stereo images. The overlapping fields of view of the two Mastcams allow stereo imaging. For single observations, the Mastcams usually acquire one full frame through each eye, but mosaics require a different strategy. Because of the different fields of view of the two cameras, it would be wasteful in data bits to capture and return to Earth full images in both eyes for each spot in a stereo mosaic. So they subframe (that is, crop) Mastcam-34 images taken as part of a stereo mosaic, returning one subframed Mastcam-34 left-eye image for every Mastcam-100 right-eye image. Operationally, the team refers to these sequences as “shrinkwrap stereo,” because the field of view of the Mastcam-34 image has been shrunk to some size that contains (“wraps”) the entire Mastcam-100 image. But because the two cameras’ boresights are toed in by 2.5° (1.5° each), the horizontal position of a Mastcam-100 image within a corresponding Mastcam-34 image depends on the target’s distance from the rover. This complicates the efforts of mission planners to determine how to subframe the images. In practice, most shrinkwrap stereo observations are cropped to match the vertical extent of the Mastcam-100 image (which is the same regardless of the distance to the target), but the horizontal image dimension encompasses all possible positions of the Mastcam-34 field of view, trading slightly higher data volume for a major reduction in planning complexity. Figure 7.3 illustrates the location of the shrinkwrap stereo subframe on the left Mastcam field of view.

  Focus stacks. Mastcam has the capability to take many images at different focal depths and merge them onboard into a single best-focus image and range map. This capability exists because it was required for the shallow-depth-of-field MAHLI and has only been used on Mars by Mastcam on two sets of observations: one on sol 193 and another on sol 1051/1052. For more on focus stacking, see section 7.4.2.3.

  Clast surveys. After drives, the Mastcams often take a stereo pair of images of the terrain to the right of the rover. These photos are shot blind and cover the field of view of the ground temperature sensor of REMS boom 1 (see section 8.​4.​2.​1).

  ChemCam documentation. Mastcam stereo images usually provide context for ChemCam laser shot points, especially blind targets and AEGIS targets (see section 9.​2.​2.​2). The ChemCam team has developed a method to automatically colorize ChemCam images with lower-resolution right Mastcam color information, helping them interpret their data.

  Multispectral imaging for mineralogy. When a target is expected to have interesting spectral content, the team uses all or some of the science filters to image it. ChemCam laser targets, brushed spots, and areas associated with drilling, scooping, or sample dumping are usually hit with all fourteen science filters. Where the team expects to see minerals that may contain water – most often found in calcium sulfate veins crosscutting the rocks – they may perform a “hydration survey” using right-eye filters 0, 3, 4, 5, and 6, or just 0, 5, and 6, to differentiate among more-hydrated or less-hydrated forms of calcium sulfate. The smaller number of filters and use of just the right-eye camera diminishes the data volume and simplifies the planning relative to fourteen-filter observations, so hydration surveys can be small mosaics without generating a prohibitively large data volume.

  When Mastcam takes multispectral images, nothing in the file names indicates which filter was used, but the filters are almost always used in order, e.g. [L0, L1, L2, ... , L6]. If it is a multi-position mosaic, the filter wheel may be spun backwards on every other footprint in order to reduce total wheel rotation, e.g. [R0, R5, R6, shift position, then R6, R5, R0, shift position, repeat].

  Photometry. Multispectral observations taken with the left camera filters 1, 2, 3, and 6 of the same spot several times over the course of a day allow scientists to study surface properties of the Martian soil. Photometry surveys are often performed when the rover is parked for some period, over holidays or during anomaly investigations. They can sequence photometry observations two or three days in a row, performing them at different times of day to build up dense temporal coverage.

  Atmospheric studies. The Mastcams routinely image the Sun through the solar filters, using the Sun’s known brightness to probe the optical depth (which is related to how much dust is in the atmosphere). Sun images are usually subframed to 256 pixels square. At the same time, photos of the sky are usually taken in a direction away from the Sun with left-eye filters 2 and 5, which have similar bandpasses to the solar filters, to measure aerosol scattering properties. On occasion, Mastcam sky surveys span the sky from horizon to zenith, with or without multispectral observations. Beginning on sol 939, they also began to take routine images pointed due north at the distant crater rim as a way to observe the dustiness of the atmosphere within the crater.

  Astronomical imaging. Astronomical imaging has a variety of scientific goals. At night, capturing movies of Phobos and Deimos passing through the field of view at the same time, or of moons occulting bright stars like Aldebaran, can help constrain the moons’ orbital positions. During the day, the Mastcams can observe Phobos and Deimos transit the Sun for the same purpose (and can even image large sunspots, from a different perspective than solar spacecraft, particularly in the Sun images from sol 1000–1047, when a very large group was visible). Mastcam has also watched Phobos enter Mars’ shadow, probing for dust in the upper atmosphere. It has targeted other bright sky objects, including Jupiter, Saturn, Ceres, Vesta, and stars like Regulus, and achieved a detection of Comet Siding Spring. Mastcams have even watched the Sun set, justified for atmospheric science purposes but mostly to produce evocative images of a sunset on another planet. Box 7.1 summarizes Mastcam imaging of astronomical targets.

  Box 7.1. Astronomical imaging with Mastcam to sol 1800.

  Sunset: 587, 956

  Transits of the Sun by Phobos: 37, 42, 363, 368, 369, 713, 1032; 1692; by Deimos: 42; by Mercury: 650, 956.

  Photography of Phobos: 45, 635, 662, 964; Deimos: 772, 777, 1732, 1738; 1742.

  Sequence of images of Phobos entering or exiting eclipse: 393, 964, 970, 979, 987, 998, 1002; 1730; 1736; same by Deimos: 995.

  Phobos over Mount Sharp at sunset: 613.

  Phobos and Deimos mutual events, 351, 378, 393, 964.

  Siding Spring: 772; with Earth, Phobos, and Deimos, 782; with Deimos 783; with Phobos 784.

  Other: Jupiter 378; Phobos occultation of Aldebaran 387; Jupiter & moons & Phobos & Deimos 393; Phobos & Jupiter & Deimos & Ceres & Vesta & Saturn & Regulus 606, Regulus 662.

  7.2.3 Anomalies

  The Mastcams have worked well on Mars, with the first puzzling problem appearing early in 2017. On sol 1576, atmospheric scientist Mark Lemmon first noticed seeing large differences between the zenith atmospheric opacity measurements computed from left and right Mastcam solar images. Yet images from the two cameras do not have different brightnesses when the rover was looking in other directions through different filters. Whatever caused this change in b
ehavior happened at some time between sol 1490 and sol 1576.8 At the time of writing, the best explanation appears to be that sand has blown into the right camera baffle, which flows onto the cameras’ front windows when the rover looks up. The Mastcam team is working on testing this hypothesis, taking some images looking inside the Mastcam sunshade on sol 1749.

  7.3 MARDI: MARS DESCENT IMAGER

  The Mars Descent Imager (MARDI)’s intended purpose was to help the science team rapidly identify the location of Curiosity’s landing site. The images would also bridge the gap between the orbital coverage of the site and the Mastcam view from the ground. MARDI functioned as designed, taking 622 images between heat shield separation and touchdown, and many more after (see section 2.​3.​7). By the time Curiosity was launched, the sharp eyes of the HiRISE camera on Mars Reconnaissance Orbiter had made MARDI’s landing-site-localization function mostly redundant, but the video that MARDI returned during the descent provided engineers invaluable information on the dynamics of the landing, and provided rover fans with a thrilling movie.