MR. MONAHAN: Good morning.
I would like to start my presentation today by thanking the panel for taking time out of their busy schedules to have a look at the material that has been submitted by G.E. and to come here today to help us in our deliberations to bring this product to market.
This, I feel, is a really important step along the road that we have taken with digital mammography that we are here today to actually look at an application and to reach some decision. During the course of the review, I would like to point out that we have involved not just the Office of Device Evaluation but also the Office of Surveillance and Biometrics in the Center, the Office of Compliance and our Office of Science and Technology.
You will notice that the manufacturer, when they got up today, used soft copy for display of their slides. FDA, on the other hand, is using hard-copy display. I don't want to panel to read anything into this about our distrust of technologies. But we are relying on the old technology here today.
I had the overall lead of this review but I was assisted by many people from the Center and I would like to thank each and every one of them for promptly giving their reviews and cooperating in this joint effort.
For the manufacturing review, we had Falidia Farrar from the Office of Compliance. There are no major problems remaining with the manufacturing aspects of the submission. The labeling has been reviewed by Dr. Sacks, Phillips and Mr. Doyle who is the Executive Secretary for the panel. There may be some lingering issues relative to the labeling which is typical for a PMA, and the agency will work those out as we move along in the process. Most of those usually consist of editorial changes rather than anything of substance.
The clinical studies and the statistical work in the application were reviewed by Dr. Sacks, Wagner and Bushar. The engineering and physics were reviewed by Robert Gagne, Robert Jennings and Kish Chakrabarti. I forgot to mention, Kish is with the Office of Mammography Quality Assurance and I didn't mention them when I was talking about offices. I apologize for that.
The disinfection and sterilization issues associated with the device were reviewed by Cathy Nutter. Again, there were no significant issues with the disinfection of the device. The information provided by the company is adequate.
We will begin this morning with Robert Gagne discussing the physics. You have heard some of that discussed earlier. This will be from the FDA perspective as will all the other presentations given today. As you are aware, the FDA had a slightly different perspective on applications than manufacturers, typically, and, hopefully, we come to agreement.
The clinical study and the statistics will be reviewed by Harry Bushar. Robert Wagner will give a semi-tutorial and then discuss some of the clinical data as it all relates to ROC analysis. The feature analysis study, the post-approval study design and, finally, the labeling will be discussed by Dr. Sacks.
We will start now with Bob Gagne.
DR. GAGNE: Good morning.
My name is Bob Gagne. I work in the Office of Science and Technology here at the Center. My job today is to go ahead and try and give you a review of some of the key aspects of the physics that are present in this particular submittal.
As a start to this presentation, what I would like to do is just give you and overview of where I am going with the presentation. Basically, what I would like to do is to quickly review for you what it is that we look for in terms of physics whenever we get an application in this manner. I am going to spend some time, and this will be a little bit redundant with the manufacturer's presentation but I think I am giving a little bit different view here so, hopefully, it will increase the knowledge a bit. That remains to be seen.
I would like to define the DQE and show you a little bit its relation to imaging performance because we are going to talk about DQE data that the manufacturer has presented in their application.
I am only going to review some of the key data. The key data is defined, basically, by me in terms of the review of the physics--we are not going to go over all the physics aspects here from the PMA--and then give you some concluding remarks.
What do we look for in terms of physics? I am not going to describe each item on this slide. I just want to say, however, that one thing that I will be doing is that the things that are in italics and the bolder color blue we will talk about some more as we go along in the presentation.
There are basically three major areas that we look at when we look at the physics for this type of device. The breakout of two of those areas are titled "detected data" and "display data." It is kind of a unique circumstance for a digital detector that, in fact, you can break those out, you can get parameters that are strictly related with detector and you can get parameters that are strictly related with display.
So we itemize those kinds of parameters and they are all in the sponsor's application. That is different than the analogue system film screening that incorporates, basically, the display in the imaging system.
Let me go on to the next viewgraph. I would like to take a little bit of time here going over this slide. I wonder if you would make the translation for me here as I talk about DQE later on in the presentation that what I mean by DQE is the ability of the system to transfer information that is available at the input to the output.
It is defined in terms of signal-to-noise ratio, but it really is its ability to transfer information. So when I say that the system has a particular DQE value, what I am saying is that I am making some value judgment on how well it is able to transfer that information.
It turns out that, if you look at the first equation here--you saw this equation previously in the GE presentation--its DQE is a measure of system efficiency in terms of how much signal-to-noise ratio squared you had into the system compared to what you get out. It has a spatial frequency dependence. That is the (f) means.
You can express the DQE in a different manner when you look into the expression for signal-to-noise ratio. It turns out that you can describe the DQE in a different manner as the ratio of noise-equivalent quanta as a function of spatial frequency to the number of input quanta.
That is interesting because noise-equivalent quanta is made up of, and I hope you can see the light color blue there--noise-equivalent quanta wraps up three important imaging parameters for imaging systems and that is its gray scale transfer, in the large G, its resolution as measured by modulation transfer function and the noise in the system as measured through a noise-power spectrum.
What I have tried to do on the right-hand side with a set of images that I think some of you probably have seen before is if you think about noise-equivalent quanta as a measure of the amount of detected X-ray photons by the imaging system, the set of black-and-white photos there represents a set of images where that number of quanta is increasing when you go to the right and it is increasing as you go down the page.
I would like to focus a bit, just to give you sort of a practical description of this concept, at the two middle pictures. If you look at the right-hand side photo in the middle row, and assume that that would be the picture that you got if you had a perfect detector, a DQE equal to 1.0.
The image to the left of that represents what would be at the output of the system if the DQE were somewhere around 15 percent. So you see the differences, then, in terms of the transfer of information and what this quantity represents.
One key piece of data that I want to bring up for you are the values of DQE for the sponsor's imaging system. I would like to spend just a little bit of time talking a little bit about the impact of design on these DQE values.
You can trade off certain aspects because of design constraints with respect to DQE. In the final analysis, what you would like to do is you would like to meet--if you look at the graph on the right-hand side--you would like to get the DQE value to go up in magnitude and over to the right in terms of spatial frequency. You would like to increase its band width if you want. Those are the things you would like to do.
But there may be circumstances where you might trade off one versus the other. One situation where that occurs is the choice of input phosphor. But I want to focus more on the size of the pixel.
There have been some recommendations in the literature, informally, about the size of the pixel. Should it be 0.05 millimeter, 0.1 millimeter, or 0.15 millimeter? That is a difficult question to answer because choosing one of those sizes involves tradeoffs.
The Senographe 2000D has a 0.1 millimeter pixel size. Now, the immediate impact of that that I think I will show you in some of these slides is that you do get some tradeoff in terms of the band width of the DQE because of the size of the pixel, but you pick up other aspects in terms of image display because the total number of pixels is smaller.
So those kinds of tradeoffs, I think, make it difficult to make a definitive statement about pixel size.
Let's go on to the actual data. This is another slide that I would like to spend a little bit of time explaining because the same motif will follow through in the next three slides. I am going to start from the top left, work my way over to the right and then down to the actual data.
First, let's consider the objects at the top of the slide. I have tried to show, in a cartoon representation, if you want, the imaging of a spiculated mass which is at the center of the breast. In this case, the exposure at the detector is close to optimum for film screen, about 11 mR. This results in an image of that spiculated mass at the center of the breast.
Moving along now, I have three circles on that spiculated mass that I am trying to show represents a different amount of stress, if you want, on the imaging system in terms of its ability to image that particular structure. Starting with the top circle, which is really just detecting whether the mass is there or not, going down to the next one down which is to see something slowly changing in shape, and, finally, to the fast-changing end of the spiculation in the mass.
The arrows are intended, then, to represent this stress, if you want, how much of the DQE, how much of the information transfer is needed in order to picture these particular pieces of this cartoon representation of a spiculated mass.
Now, let's go on to the data itself. You saw this DQE route before. What I would like to do is summarize a little bit. Let me make a statement, first of all, about the film-screen system. The system that I have picked is intended to be representative of the performance of a typical film screen. I am not intending to take the absolute best, but it certainly is a good representation of the performance of a film-screen system.
Now, with respect to the graph, a couple of points. First of all, as for any digital detector, there is a frequency at which faithful reproduction of signal when you are near or above that spatial frequency is no longer possible. That is really determined by the pixel size.
For the G.E. system, that frequency is 5 line pairs per millimeter related to the 0.1 millimeter pixel size.
Now, let's look at the data, itself, and see what conclusions we can draw from this. First of all, the sponsor's system has a higher DQE for almost all frequencies up to the Nyquist. But the film screen has response, transfer of information, DQE beyond the Nyquist frequency.
So, with respect to those particular imaging tasks, then, I hope this gives you a bit of a feeling as to the advantages and disadvantages for these systems at this particular operating point, 11 mR.
In the next slide, I won't go back in terms of saying what is going on with the imaging task. What has changed in this particular slide is the exposure to the detector. We are talking, now, about a situation where we have a mass near the skin line. The higher exposure, in this case, 22 mR, is intended to show the conditions of the detector at or near the skin line.
Now, if you look at the DQE for the sponsor's system, you see it is quite a bit higher than film screen. Film screen has fallen off considerably. Again, there is no response for the digital beyond five linepair and there is a little bit for the film screen.
So, in thinking about the future analysis, I think this particular graph, to a certain extent, explains some of those results.
At the other extreme, suppose we are in a region of the breast which corresponds to a dense area of the breast, now the exposure at the detector is less than the typical 11 mR. It is 1 mR. Again, we see similar characteristics. The sponsor's system has higher DQE values on the order of two to five times than the screen film and it stops at 5Êlinepair per milligram per milligram.
So, in summary, then, at the risk of being a little bit repetitious here, what I am saying is that the DQE for this system, for exposure which is close to optimum for film screen, indicates that the DQE for the Senographe is higher than film screen almost all the way up to the Nyquist frequency.
It is a digital detector so a faithful reproduction of signal is not possible near and beyond the Nyquist. As far as conditions of exposure that are near a skin line or in a dense area of the breast, we saw that the transfer of information, as measured by DQE, falls off considerably for film screen and the digital system remains high on the order of two to ten times higher than the film screen.
So you get a significant increase in dynamic range. I am talking dynamic range in terms of transfer of information here for the applicant's imaging system.
Let me talk about a couple of other key components associated with this type of imaging system. If you think about the major contributors to noise in these systems, there are two major pieces. One is the quantum noise that comes strictly from the X-ray photon statistics. But then there is also additive noise from the detector in the electronics.
What you would like to have in an imaging system is you would like to have the total noise be dominated by the X-ray photon statistics, not by the additive noise of the electronics. You would like to have this quantum-limited operation over a range of exposures that are appropriate for mammography.
So we are looking at this particular parameter because of this characteristic--you want this to be dominated by quantum noise--and because, formally or informally, there have been circumstances where sometimes the electronics are, in fact, quite noise.
If you have significant additive noise, it will have an impact on this summary measure, DQE. The impact that will be such that it will impact the value of DQE at low exposure values.
So, going on to the sponsor's data, now, you saw this graph previously. This is a different graph than what I had before. Previously, the abscissa represented spatial frequency. Now I am showing you the value of DQE at a particular spatial frequency, 2 linepairs per millimeter, as a function of exposure to the detector.
The DQE is essentially flat until you reach exposure levels on the order of about an mR or less. So the significance of the additive noise doesn't come in until you are almost out of the range of operation for mammography exposures. As a comparison, I have shown you a film screen plot for the same exposure, the same film screen that I was showing you before.
In this particular case, what dominates the noise on the low and high exposure for film screen is not, of course, electronic noise but additive noise brought out by the film grain. And so when the relative contribution of film grain versus quantum noise starts to be large, the film screen's DQE or transfer of information goes down.
As you can see, this particular system, G.E. full-field digital mammography, at this spatial frequency, outperforms the film screen.
Going on to a couple of other datapoints with respect to the physics, there is image conditioning and display which is going on with respect to the digital data. Some of this conditioning involves the thickness compensation so that when you look at a laser-film-recorded image of a breast from the digital system, you don't see the wide range and optical density that you would see in a regular analogue film.
There is processing going on. There is linearization associated with perceptual linearization for the display device and linearization on the device, itself. All of this is conditioning associated with getting a final display on the laser film recorder.
There is really not very much consensus or standards on relating necessary performance levels for these display devices, whether it is soft copy or, in this case, we are talking hard copy to the characteristics of the digital data. In our view, in looking at the submission, the steps that have been taken seem reasonable and appropriate.
But, in the final analysis, at this point, we really have to rely on the demonstrated clinical performance associated with the protocols and the algorithms that are being used for conditioning and display.
Lastly, one aspect that is unique to visual detectors is the fact that you can have artifacts on the image that come from bad or defective pixels. The manufacturer specifies limits with respect to these bad and defective pixels. Again, there are no standards or guidelines. There is no consensus here with respect to pixels, bad pixels.
So what is reasonable is really somewhat up in the air. Not only is what is reasonable up in the air but there are no requirements to provide any information in terms of where the bad pixels reside with respect to the detector.
Just to go over a couple of the criteria that are used by the sponsor in this area, bad pixels, before you correct them, a lot of these pixels can be corrected. The tolerance is being specified as a maximum of 1100 isolated pixels or pixel pairs--this is a maximum now--and no large clusters--that is, you can't have any large clusters of greater than, for example, 15 or more adjacent pixels in the line.
After correction of these pixels, you can't have more than one bad pixel in any 2 centimeter by 2 centimeter area. Again, as I said, there is no consensus here but it looks to us, in terms of these tolerances, that these are reasonable tolerances for this particular kind of device.
In conclusion, the data pertaining to the physics aspects in the PMA I think provides important information on comparative imaging performance between a digital system and its analogue counterpart and actually between other digital systems, if you want, also.
System parameters like DQE and quantum-limited operation provide the means to evaluate the advantages and disadvantages of the different imaging modalities. There is a summary of this data in the labeling and so, looking at it in terms of adequacy and availability, this data is in the labeling of this particular device.
We think, and it is my opinion, that this sort of information is not only appropriate for the device labeling but can also serve in the future as a point of reference for the community.
DR. GARRA: Thank you.
The next speaker for the FDA is Dr. Harry Bushar who is going to be talking about the statistical review of the clinical data.
Statistical Review of the Clinical Data
DR. BUSHAR: Good morning.
My name is Harry Bushar. I will be doing the statistical review. I looked at what the sponsor had presented in their clinical trials and what I will be presenting is my review of the sponsor's analysis.
I focussed primarily, or entirely, on the second reader study for the simple reason that this study was done a little bit better in that all five radiologists read all of the mammograms from all of the women. The sponsor compared the digital mammography to the screen-film mammography in a clinical trial which consisted of 625 women.
There were 581 from a diagnostic series that did not have cancer. There were 24 in the series that did have cancer. And then there were 20 women with cancer taken from a screening series. Each women received both a two-view screen film and an equivalent two-view digital which was performed using technique factors that were matched. Notice the digital was matched to the screen-film technique.
In the second reader study, the diagnostic cohort consisted of 605 consecutive women who were attending for diagnostic mammography at four sites, one in Colorado, one in Pennsylvania and two in Massachusetts. The screening cohort consists of 20 cancers--that is, the first 20 cancers--selected from approximately 4,000 women in an ongoing screening study which was conducted at two of the above four sites, namely Colorado and one in Massachusetts.
In the second reader study, the sponsor used five MQSA-qualified radiologists to independently interpret each digital and each screen-film mammography which were obtained from a total of 997 breasts from the 625 women enrolled. Some women only had mammography done on one of the breasts.
The digital images were stored digitally and laser printed for reading. The printing was done at the Colorado and Massachusetts facility to provide the comparability to screen film; that is, everything was hard copy in this particular reader study.
What I will be looking at here is patient management. In other words, I am going to look at the ACR BIRADS categories which were defined to be negative when they were one, normal, or two, benign, for breast cancer for both the screen film and the digital. In the other ACR BIRADS categories, namely 0, needs further evaluation, 3, probably benign, 4, suspicious of breast cancer and 5, highly suspicious of breast cancer, are all considered positive. This was done for both the screen film and the digital, so my sensitivity and specificity will be relative to these definitions.
The specificity, or 2 negative rate, was estimated by the sponsor for digital to be 55 percent. It was slightly numerically larger than the corresponding estimate of screen film which was 53 percent. The sponsor did an equivalence test where he looked at the difference delta between the digital specificity and the screen film specificity. He used a model--he used a SAS PROC MIXED and he was able to adjust for the fact that there were five readers for each mammography to obtain a confidence interval, a 95 percent confidence interval, for this difference which extended down as far as -0.6 percent up to about 4 percent.
He was also able to reject his equivalence null hypothesis. The equivalence null hypothesis was that the delta would be less than -5Êpercent; in other words, the digital would be worse in specificity than the screen film by more than five percentage points. This was done with a p-value of 0.001, so it was a highly statistically significant result in terms of equivalence.
Correspondingly, he looked at sensitivity, or the true positive rate, and his estimate of digital sensitivity was 68 percent which was now slightly numerically smaller than the corresponding estimate of screen film which was 70Êpercent. Here, I have presented the delta in the same form as I did for specificity, so as not to be confusing. But the sponsor looked at digital sensitivity minus screen film sensitivity and used the same type of model in the SAS PROC MIXED to take care of the correlation between the multiple readers and obtain the 95 percent confidence interval now that went all the way down almost to
-10 percent and up to 7 percent.
But, still, he was able to reject the equivalence null hypothesis that delta was less than -10 percent. In other words, he rejected the null hypothesis that the digital sensitivity would be worse by ten percentage points than the screen film sensitivity, but just barely because the p-value now is less than 0.03.
So, therefore, in conclusion, the sponsor's second reader study demonstrates that for patient management in a diagnostic population which was enriched with cancer selected from a screening study that the digital specificity is not lower than 5 percentage points below the screen film specificity and also that the digital sensitivity is not lower than ten percentage points below screen film sensitivity.
We have to realize, here, that there are some biases because we are dealing with a diagnostic population and there may even have been some bias in favor of analogue because the women, perhaps going to the digital clinic, had been screened previously with analogue.
But the way the sponsor did this study, they tried to minimize, if not eliminate, this bias. They took consecutive women showing up at the diagnostic center so that each women that was selected for the population received both an analogue and a digital. That analogue was not used to select that woman for the study.
In the screening study, all women who entered the study received both digital and analogue so there was no obvious bias on that study.
That's it. Thank you.
DR. GARRA: Thank you.
The next speaker for the FDA is going to be Dr. Robert Wagner who is going to review some of the ROC analysis features that are found in this study.