Radiological devices panel meeting

Download 0.49 Mb.
Date conversion15.05.2016
Size0.49 Mb.
1   2   3   4   5   6   7

Panel Discussion

DR. DESTOUET: I want to thank the manufacturer and Dr. Hendrick for excellent presentations this morning as well as the FDA. With that, I have a couple of questions for you, Dr. Hendrick, if you will approach the podium.

In the design of the study, the manufacturer chose to select diagnostic mammography patients enriched with a number of cancers from previous screening programs and tell the readers that they had to read them as though they were screening mammograms. It seems that there is, indeed, certainly an inherent bias toward probably larger lesions in that population because there would be a certain number of patients with palpable lesions who you would not expect to have in the screening population.

I just wonder why did you choose that as opposed to having a screening population as one of the two series of studies?

DR. HENDRICK: Primarily because we set up the patient recruitment for this study based on the guidance the FDA put out in '96 which was based on an equivalence study in the diagnostic cohort. So we began with that approach. And then, when the meeting occurred in August of '98, it was clear that an equivalence approach wasn't feasible and it was switched to a noninferiority approach. We didn't want to throw away all the patients that we had recruited.

In response to your mention about it being more biased toward larger lesions, that is what we would have expected from a strictly diagnostic cohort but, in fact, the stage and size information suggests that it was remarkably close to a screening cohort in that distribution, probably somewhere in between but close to a screening cohort in terms of stage of detected cancers, at least.

DR. DESTOUET: The recall rate of 44 percent and higher clearly was significantly much higher than one would see in a screening population.

DR. HENDRICK: Absolutely.

DR. DESTOUET: Where your recall rate would approach 10 to 15 percent. Do you have any data to show that the lesions that were recalled from screen film were the same lesions that were recalled on the full-field digital images?

DR. HENDRICK: We have the data in terms of the two-by-two tables for recall rates that are in the documents that you have. If I get my document, I can point to the table.


DR. HENDRICK: It is table No. 24.

DR. DESTOUET: What page?

DR. HENDRICK: It is page 0108 in the larger printed numerals. It is table 24 in the study report. If you look at, for instance, the noncancer cases, and this is the composite results of five readers on 625 cases, so, out of 3125 readings, there were 936 cases or readings that were read positive on both and a total of about 800 that were read positive by one but not the other.

So there was a considerable amount of disagreement between the two modalities. That sort of is an indication of why the equivalence approach, as originally formulated by the FDA, was not achievable. There was agreement on 1150 being negative on both.

DR. DESTOUET: You also mentioned that there were lesion markers in some cases and you had to eliminate those cases because of "lesion markers." I didn't understand.

DR. HENDRICK: Oh; it was simply when you put the film-screen and digital cases side-by-side, some institutions marked lesions with PVs on the films if they were palpable lesions and some institutions--most institutions do that, but what we found is, in a few cases, there was a marker visible in one modality that wasn't visible in the other.

So we eliminated those cases where there might be more of a suggestion of a finding in one modality than another. If the lesion markers were equivalent in the two, we kept them. But it was just to make the study as pure as possible in terms of avoiding any kind of bias toward one modality or the other.

DR. DESTOUET: Ed, as you look at the technology, do you feel that the difference in position that has been described, that can, indeed, obscure some lesions, whether they be malignant or benign, is such that it would be difficult to really compare the two modalities in any kind of study?

DR. HENDRICK: I agree with the speakers that have gone before that have pointed to the reader variability as being a big issue. The positioning variability, I think, is a bigger issue than we have given it credit for, not so much in that you do see a lesion clearly in one positioning and in the same positioning with the other modality you don't see it clearly. It is the little signs that throw it from being, say, a 1 or 2, usually a 2, into being something that would be called back, a slight sign of spiculation, something like that that is visible in one position and not visible on the other modality in the corresponding position.

There is no reason to believe that you would get different results even if you were using a single modality and repositioned the patient. So I do think that has an effect. And, certainly, the reader variability has an effect on not being able to exactly compare these modalities without those sources of variability.

It just makes it much harder to do reader studies because of that.

DR. DESTOUET: Do you believe that the flexibility that we will have with manipulating window settings with full-field digital will, indeed, offset some of the problems that we are seeing with, perhaps, changes of position? Is there anything in this new technology that will help us to eliminate women having to come back for call-back?

DR. HENDRICK: Yes; the data that we have so far that you are looking at are all, obviously, hard-copy interpretations of full-field digital. The data that you don't have in front of you are the data on soft-copy interpretations of full-field digital. My experience with that comes from data in the Army study at Colorado and U. Mass. There we also see a significantly lower recall rate with digital primarily because the radiologists do have the flexibility of the soft-copy display to window through the lesions and they are not stuck with sort of the opaque glandular tissue that you sometimes have on film-screen mammograms.

You can window through and better visualize the extent of the lesion than you can on a fixed hard copy, whether it is on digital or film screen. So I think that will offer some real advantages if radiologists are able to do soft-copy review of digital mammograms.

DR. DESTOUET: I want to open it to the panel now. I have some further questions but are there any questions from the panel?

DR. BERG: I have a question that is kind of a two-parter. I think one of the issues, obviously, that will be the subject of additional discussions as we get other applications from other manufacturers will be the issue of the resolution. I know G.E. has shown the 0.1 millimeter. Obviously, there are malignant calcifications, at least, that might be smaller than that limiting resolution.

So there is sort of a fundamental question out there, at least in the mind of many of us, until we have seen enough of these very subtle lesions on digital to really be satisfied.

I noticed, in looking--I will let you answer that part, but I wanted to address the ROC curve from reader study No. 1. Looking at the middle ground, if you will, in what would sort of what would be BIRADS Category 3 and 4 type lesions, it looked like full-field digital did worse than screen film.

I guess one of the possible interpretations of that, and I don't know that it is even was statistically significant if you just looked at that subset--but one of the interpretations could be that you are not seeing some of the very tiniest calcifications that might push you to be more concerned or that you are not seeing the border characteristics of a mass lesion as being clearly indistinctly marginated as opposed to being partially obscured.

Some of these issues that are right on the fringes where I think the only real remaining concern that some of us have, I was wondering if you could address. Also, then, I didn't see that difference in reader study No. 2 so I was wondering if the same readers learned, if you will, or if they were even different readers in study 2 versus study 1.

DR. HENDRICK: Right. They were different readers in study 2 versus study 1. I think you are right in seeing that in the places when the curves do differ, it is in the 2Êpercent, 3 percent probability of cancer range. The low probability--and there were cases where readers, I think, in that study were reading the digital mammograms with less confidence of what they were seeing which, in some cases, turned out to be cancer than the screen-film mammograms.

Partly, I think that was an effect of the readers not all being trained to the same level of familiarity with digital. Some of the readers started digital cold, basically. We basically had readers at two institutions as primary readers. Some had experience through reading Army cases in the Colorado/U. Mass Army Study. And then there were two other readers who didn't have that experience.

I think a lot of the differences in the curves, especially in that low probability of cancer range come from those readers that didn't have experience in the Army study. When we did study No. 2, we got everyone to the same level of experience with digital and we made sure that they were all reading the cases as if they were screening cases.

The two readers that we had that weren't in the Army study and reader study No. 1 were really reading them more as if they were diagnostic cases.

DR. BERG: I think that I certainly have concern about that issue just because a lot of the cancers that we see are not spiculated masses or obvious tracking branching calcifications. Many of them are in that middle ground. I think one of the issues that we all have to wrestle with a little bit is what kind of training we are going to require of people before they start doing this just so that we are not misinterpreting those subtle lesions.

DR. HENDRICK: Right. But if I could address the first thing you brought up about were these the cases that, for instance, might have had subtle calcifications that digital didn't see and screen film did see. Those would have showed up in the side-by-side analysis specific for calcifications or specific for other lesion types and they didn't.

The scores were virtually identical even for those different lesion subtypes. So I really think, based on the data that we have, it was a reader issue in study No. 1 and not a shortcoming of the modality in either study.

DR. BERG: Do you have any specific information on, like, amorphous calcifications, the tiniest ones, side-by-side, or any breakdown on future analysis including benign lesions in terms of conspicuity, some of these subtle ones? It is hard to answer, from what I have seen and hear, one way or the other.

DR. HENDRICK: We don't have data on the smallest calcifications detected, but I just would want to make one point which is that calcifications tend to be fairly high contrast relative to rest of the breast. Even if they are smaller than 100 microns, they can be detected in that 100-micron pixel. Where you may have a limit is in breaking it apart into more than one calcification, but you should still, with these high-contrast objects, detect things smaller than 100 microns. You just may not be able to have a good idea about the shape of the calcifications from when they are that small.

DR. BERG: Right.

DR. HENDRICK: I think there is another clinically interesting question which is how small calcifications do you really depend on detecting with screen film. If you compare specimen radiographs that are taken after the excision of the sample, you can see much, much smaller calcifications in the lesion than you tend to see on the screen-film mammograms or the digital mammograms.

So the question is is it making any diagnostic difference. Our data to date support that there isn't a difference, but I think that needs to be studied in some specific studies that look at calcifications as well.

DR. ROMILLY-HARPER: Excuse my voice, but in, I think, the last study, the laser printer--actually, the quality of the digital mammograms for the study were produced at one institution, as far as you ship them out to one place and then--correct me if I am wrong--

DR. HENDRICK: There were actually two institutions, Colorado and University of Massachusetts.

DR. ROMILLY-HARPER: But one person was in charge in printing out the digital studies for review; is that correct?

DR. HENDRICK: There was one person at University of Massachusetts. Actually several people at Colorado printed them out.

DR. ROMILLY-HARPER: Are we going to have--if this is approved, how do you plan to do that for the multiple institutions? Do you have set criteria for the copies, how the copies are going to be made on the laser printers?

DR. HENDRICK: Yes; we have actually learned a lot from doing these studies and printing them out on hard copy. There will be criteria for how to print them out. For example, one of the things that we have learned is that the way laser printers are set up, and they are based on other modalities, typically is that the sort of middle range of signal--you window and level it on the monitor and then the median grey scale on the monitor on a laser camera gets printed out at an optical density of around 1.0.

That means that more glandular tissues would get printed out at even lower optical densities on the film. We have learned, in doing this, that that makes no sense for digital mammograms, just like you wouldn't want to take a film-screen mammogram and have the average optical density of the breast be at 1.0 and have all the glandular tissues be at lighter grey scales on the image.

So one of the things that we have learned is that you need to set the median density in the laser printers to a higher level so that you have more signal values to work with in the most important part of the breast which is in the glandular parts of the tissue and in the grey scales that would represent cancers in the breast.

So there are guidelines that need to be put in place for people doing this kind of printing and it also depends on the setup of the printer with the monitors from which the images are printed. So you need to have the printed images looking like they look on the monitors that are being printed and to have advice to the people printing the images.

DR. ROMILLY-HARPER: That means that, as they purchase the unit, they will have dedicated laser printers for the digital mammography and not the typical laser printers that you see in a department. In other words, the unit will be sold with the printers.

DR. HENDRICK: In the initial digital-mammography systems that are out there, they have to use hard copy, so they have to be equipped with these printers.

This is Amy Sitzler who works with G.E. Medical Systems. She has some comments, if she may, about the QC on printed images.

MS. SITZLER: I am the Program Manager for the Digital Mammography Programs at G.E. There are two printers which are qualified to be used with the system, and only those two so far. So we don't just allow images to be printed anywhere.

We have been working with the same QC procedures as you would be used to with MQSA so that you would do the same qualifications with your printer for the digital system as you would be used to doing with your screen-film system. So you should have the same daily kind of check that is on the QC menu like you would be used to in your normal film-screen system.


DR. DESTOUET: Actually, I have a question for you. Is it anticipated that there will be one single setting, then, for the hard copy as opposed to what we have now with CT where you may have a couple of different window levels and settings?

Can we, indeed, produce a single hard-copy image that will give us the latitude as well as the resolution that we need?

MS. SITZLER: There is the capability. Because we have a lot of experience with a lot of images, we have tuned the algorithm for the automatic setting of the contrast so that the image will appear at a certain contrast setting which is selected automatically based on that breast.

It can be printed automatically based on that setting. Of course, the technologist always has the opportunity to make her own adjustments, but we have actually designed an algorithm that allows the optimal setting immediately on presentation of the image, and that is automatically sent to the printer.

DR. MALCOLM: I just had a question. I looked under the training program--we were talking about, as we always talk about in issues like this, readability and variations. It was unclear to me the QA or QC program for the radiologist in making sure that he or she has the proper tools to understand the digital radiography. I wasn't quite clear. I didn't see much of that in this proposal.

What I am saying is you are not going to sell the units and say, okay, go out here and just start using it. What is the plan?

DR. HENDRICK: MQSA requires eight hours of training specific for radiologists on digital mammography. One of the ways to satisfy that is the plan to set up a training facility actually at Northwestern to provide those eight hours of training to the radiologist. Part of it would be--a large component of it, actually, would be working with the radiologists there who have experience reading digital mammograms and looking at the presentation of digital mammograms with the thickness equalization applied both on film and on soft-copy display so that they are familiar with the kind of presentation.

It is not that it is that different from film screen, but they need to see an adequate number of cases where there are subtle cancers presented with digital images.

The artifacts are different. One of the great things about digital is you eliminate the artifacts due to the processor on film-screen mammograms, so those artifacts are basically eliminated. There are some other artifacts that come if the digital detector isn't performing properly that need to be recognized.

There are artifacts that come if the printing isn't occurring properly that need to be recognized. So the training would include recognizing those kinds of features in the image and then understanding the process by which the images is created and produced.

So it is to give them a broad picture but to focus on interpretation of both hard-copy and soft-copy digital mammograms. I don't know if I have given you enough information or not.

I don't think Northwestern will be unique in offering the training of radiologists in this. There also needs to be training for the physicists and technologists that will have to be provided as well.

DR. DESTOUET: Dr. Harms, do you have any questions?

DR. HARMS: No; not at this time.

DR. GARRA: I have a couple. The first question I would have, in study 2, you talked about testing nine radiologists and picking five of those.


DR. GARRA: Could you tell us a little bit about what selection criteria you used for that?

DR. HENDRICK: One of the things we learned in our first reader study was that there was a wide range of sensitivity and specificity among the different readers. Part of it, actually, had to do with this difference of two reviewers reading the images more as diagnostic studies and others as screening studies.

What was done is to--Craig Beam has developed a test set of images. In fact, the ROC data were presented here from his test set and there were something like 108 datapoints. So we used his test set to test these nine readers.

The goal was not to take the best five out of those nine but to find readers that had a similar sort of range. It was really to eliminate those that were not good readers of mammograms at all. We actually, I think, threw out one reader at the very top of the ROC scale and three readers at the bottom of the ROC scale to get the five that we picked, in terms of ROC areas.

DR. GARRA: Another question I had was you asked the readers to rate the probability of cancer being present. I see that on your form.

DR. HENDRICK: The form does not mention that they should not be rating BIRADS categories 1 and 2, yet you mentioned that in your talk, I believe, earlier. And even if you do eliminate BIRADS 1 and 2, what assurance do you have--why did you eliminate BIRADS categories 1 and 2? Because a person classifies something as BIRADS 2 does not mean they are 100 percent certain.

So I am just wondering if you created a little bit of a bias by excluding those people, the ratings from those people.

DR. HENDRICK: I would appeal to the radiologists on the panel. Would anyone give something, even a 1 percent probability, of cancer if you called it a BIRADS 1 or 2?


DR. HENDRICK: That was the reason for that.

DR. GARRA: Okay. That is what I wanted to know. Did you get ratings for those categories, though, at all?

DR. HENDRICK: In terms of probability of cancers?


DR. HENDRICK: Let me ask Karen White who works with MedTrials to answer that because she collected all these data and I get confused between three studies, these reader studies and the Army study.

DR. GARRA: When you talk about asking somebody to make sort of a binary decision, there is a big difference between getting him to grade on a grading scale versus a binary decision. People will often make a decision, this is absolutely not cancer, but when you actually pin me down, it will probably be something like 98 percent or 99 percent.

I think a lot of radiologists, when they think in terms of categories, 3 or 4 or 5 categories, think in terms of binary decisions, what am I going to call this. But when you ask them to really nail down percentages, they might answer slightly differently.

I just wanted to see what your numbers actually were.

MS. WHITE: Karen White. I work with MedTrials. We were the company that G.E. asked to help with the monitoring and project management for the collection of the clinical data.

In the first study, the collection of probabilities of cancers was something that, with the changes in study design through the different means of the FDA that evolved. So, in the first study, we only collected probabilities of cancers for BIRADS 3, 4, 5 and 0. It was an assumed, and correct me if I am wrong, probability of cancer of 0 for BIRADS 1 and 2.

And then, in the second study, is the one where we did collect. We asked them to give a probability of cancer for any BIRAD. Part of the instructions to the radiologist that Dr. Hendrick and Dr. John Lewin also provided to the radiologist was based on ACR categories of how you should grade probabilities of cancer, so it was based on ACR BIRADS recommendations.

DR. HENDRICK: So I guess we did collect in reader study No. 2. I had forgotten that. Is that summarized anywhere?

MS. WHITE: Yes; on page 0165 under the clinical study, summary No. 2. It is under tab D.

DR. HENDRICK: That is the form, but do we have results? But do we have results? I guess the question is did anyone who gave it a BIRADS 1 or 2 give it anything other than a 0 probability of cancer?

MS. WHITE: We had a couple, I believe, that were between 0 and 2 percent.

DR. GARRA: I presume that those people did not figure into your--were they assigned the number they actually were given or were they assigned 0?

DR. HENDRICK: No. In reader study No. 2, the analysis was done based on what they gave.

DR. GARRA: Okay; great. Thank you.

DR. DESTOUET: Are there any other questions from the panel? Dr. Toledano?

DR. TOLEDANO: So when you set up the studies, you made a choice that had great public-health impact. You chose a delta of 0.05 for your recall but a delta of 0.10 for sensitivity and ROC curve area.

Can you explain those choices, what motivated the difference in the criteria?

DR. HENDRICK: Partly, that was motivated by the understanding that we don't have as good a determination of sensitivity or ROC-curve area as we do of recall rate due to the numbers involved in those categories. The categories, specifically, for sensitivity, you need number of cancer cases, as you know. And, for ROC curve, the power depends largely on the number of cancer cases as well.

So we recognize that, without doing an immense study, we wouldn't have the ability to refine the delta as well as we could for the recall rate.

DR. TOLEDANO: So if you were approved and went into your postmarket study, you would be looking for smaller deltas.

DR. HENDRICK: Yes; the suggestion, based on what Bob Wagner presented, is the delta on the ROC-curve area would be closer to--would be; not closer to--would be 0.05. The study design would be set up to be able to see a difference as small as 0.05.

DR. TOLEDANO: Thank you.

DR. HENDRICK: In a non-inferiority approach.

DR. TOLEDANO: Correct.

DR. HARMS: The false negatives would be a patient that you did not see a lesion on a study and then, subsequently, found either a lesion on the other study or on your follow up. How many cases did you have that had false negatives on both interpretations and that you caught on the one-year follow up?

DR. HENDRICK: That is in one of the tables in the study report. If you look at, say, study No. 2--table 26 on page 0110. This is, again, five readers reading the 44 cancer cases, so there are 220 readings. There were 36Êreadings and this table only includes cancers so there were 36 readings that were negative on both modalities.

DR. HARMS: But that doesn't say that they were false negatives.

DR. HENDRICK: Yes; those were false negatives. In fact, if you want the false negatives for screen film, you sum the column that was read negative on screen film. If you want the total number of false negatives for digital, you would sum the row across the bottom as the total number of false negatives for digital.

MS. PETERS: This is just a little different focus. I notice that the device is being used in a number of different countries, or other countries. Do you have any data, or any information, from those countries about how they are experiencing the equipment?

DR. HENDRICK: I think I need to defer to the G.E. people on this one because no one lets me out of this country.

MS. SITZLER: Could you restate your question? I am not sure I understood exactly what--

MS. PETERS: Just is there any information or any data from any of the other countries that are using the device.

MS. SITZLER: So far, the device is installed in ten different sites in Europe and the data is--we haven't done this kind of detailed analysis, but the data is consistent with what we found already in this study.

DR. SMATHERS: A follow up on that. You say it is used in ten other sites. Are they using soft copy readout or are they constrained to this hard copy readout that has been proposed here?

MS. SITZLER: They are not constrained and they do both. It seems to be a site preference and a learning curve. But they are very much using soft copy.

DR. SMATHERS: Can I pursue this? Ed, I am a little concerned that you are essentially releasing this device in what I see as its least favorable light. I think hard-copy readout puts it in the worst-case scenario as far as its capabilities go. I guess I am troubled by the fact that you are not going to put soft copy with it initially because of the greater flexibility in windowing and so forth that would give the radiologist a look at a given mammogram.

I would like to have some insight as to why you chose that other--and statistically what you did made it easy for the statisticians, but actually I don't think it makes good medical sense.

DR. HENDRICK: No; that wasn't the main reason. The main reason was based on earlier advice from the FDA, specifically the guidance document that, in writing, said these proposals--the digital mammograms have to be done in hard copy. Part of the concern I think they had justifying that point was that they wouldn't have a record of what the radiologist looked at in a soft-copy display of the digital images.

If hard copy were used, they would at least have a record that they could go back and look at to say, this is the way the image was displayed to make that interpretation.

I agree with you that the true flexibility of digital and the true benefit of digital is primarily realized in a soft-copy display of the images. I think the plan would be to move very rapidly after approval of digital and hard copy to proceed to this comparison study of hard copy and soft copy to make sure that digital was available to radiologists with soft-copy interpretation.

Scott, do you want to--

MR. DONNELLY: I think that is exactly right, Dr. Smathers. The genesis of the study originally, since it was initially conceived as an equivalency, meant that we had to take the digital and turn it into an the equivalent medium in order to have a fair equivalency study and not to have the media be the difference between the studies, to eliminate that.

But I think you are absolutely right. In fact, a question earlier about window leveling and all the various techniques which you would expect today in doing, say, a CR or mR review, to lose those degrees of freedom in a mammogram exam certainly takes the digital and, to some degree, levels the playing field, if you will, with hard-copy review to conventional film screen.

That is why we have proposed that the first thing we want to do is do the soft-copy amendment to the PMA, get that passed, before we proceed with the broader post-clinical trials because I think, in order to really fairly compare and see the effectiveness of the device, it is going to be much more clear in a soft-copy environment than a hard copy.

I think, however, what we have shown is that, in a hard-copy environment, it is at least reaching the same levels, at least as the clinical data showing the same levels, of effectiveness as a film screen. But we would also expect it to be much better in soft copy.

DR. TOLEDANO: More on the public-health questions. In the request for expedited review, G.E. notes that we would expect wider patient acceptance because there would be shorter exam times. Also, I notice that there might be a decreased need for additional magnification views.

In light of the fact that the machines and the systems are being used in other countries, has that been their experience?

MR. DONNELLY: If I could comment on the other country installations. We have ten sites. The product has been in production in non-U.S. countries for a very short period of time so I think that, in all fairness, at this point, that we would say that we have any statistically relevant data from those sites would be presumptuous. So it is installed in other countries where they have already passed the regulations, but I don't think I would use that data for purposes for our approval at this time.

DR. TOLEDANO: What about anecdotal data from your previous studies in the states? Still just anecdotal?

MR. DONNELLY: Again, I think, at that point, we would probably prefer to call on radiologists that have actually been using it, some of whom have used it in the soft-copy evaluation review. Anecdotally, we have certainly had very positive feedback but I think I would have to defer to the radiologists that have actually used it to make a fair assessment.

DR. TOLEDANO: Thank you.

DR. HENDRICK: Part of your question was about the speed of doing digital acquisitions. One of the steps that is eliminated is that the technologist taking the film-screen cassette after, say, four films are taken, walking to the processor, putting them, one-by-one, through the processor by whatever means and then waiting for them to come out of the processor.

The acquisitions can take place as quickly as every ten seconds for different views on the system and the images pop up, and somebody is going to have to help me here, in less than ten seconds after the exposure is done. So that speeds that part of the process and that is part of the reason that it will speed the overall acquisition of images and increase the throughput.

DR. BERG: I have a question. You are asking for approval for diagnosis. From the data presented here, you were presenting screening views, what amounts to screening views to your readers. We don't really know, with spot magnification views done on a digital unit or with additional spot compressions done on a digital unit, that the readers would have reached the appropriate conclusion to biopsy the lesion. Am I wrong in that statement?

DR. HENDRICK: We don't have direct data here on the performance of digital in that spot magnification mode. There are data collected but not in the PMA application for that. Northwestern has done a big study of digital versus film screen for workup but the expectation is that the unit, the small focal spot, is the same as the DMR unit which is very good for film-screen spot magnification.

The magnification stands are essentially the same as the on G.E. DMR. The only replacement is the digital image receptor replacing the film-screen image receptor. So the expectation is that actually digital will do even better there than film screen because the only change is the change in the image receptor and you are spreading the lesion out or the calcifications out over more pixels in that situation.

DR. BERG: I think one of the other issues is that you are also asking for approval in the labeling at least for screening and, yet, we are being presented with what amounts to data from a diagnostic trial. I don't know for sure that these are big problems but I am just trying to go from the data that we are being handed today to review a consideration of this approval.

DR. HENDRICK: My only comment on the is that as far as the evaluation of the two views of each breast, you are doing essentially the same thing in screening or diagnostic. The only motivation for using the diagnostic population was so that we wouldn't have to image thousands of women to get an adequate number of cancers to be able to validate the device.

When you turn to a screening population, you can expect 5 per 1000. So, to get 40 cancers, we are talking 8,000 women at least imaged with both modalities.

DR. BERG: I guess I was a little surprised just because we are all familiar with John Lewin's presentation and the recent Diagnostic Imaging article. I think there is a lot of data that G.E. has collected and, as part of the Army trial, I guess I would have appreciated seeing some of that as part of this application.

But I don't know enough about the entire logic that went into that.

DR. HENDRICK: Well, that is not G.E.'s data. That is an independent study that is not funded by G.E. or in any way affected by G.E. I think the idea is to keep it that way. The images are read completely independently of this. It is funded independently of G.E.

It was a concession to get the twenty cases that had cancer for inclusion in this case but, going back to the original guidance, it was to evaluate a diagnostic population. That is the course all the manufacturers had been instructed to embark on and that is what was done here.

DR. DESTOUET: Are there any other questions from the panel?

DR. GARRA: I have a couple more.

DR. DESTOUET: Yes; go ahead, Brian.

DR. GARRA: This has to do, again, with the work station and image processing. The first question I have is the work station that you are supplying, the so-called non-diagnostic work station is the Advantage Windows platform. I just want to know if that is the system you are supplying overseas for soft-copy reading or are you supplying a different system?

MS. SITZLER: It is identical. It is not the Advantage Windows system. The platform is the Advantage Windows platform. We built a specific mammo application on top of that.

DR. GARRA: So you are basically running that SunSpark station, then.

MS. SITZLER: It is the platform for both work stations.

DR. GARRA: You are talking about hardwarewise?

MS. SITZLER: More softwarewise than hardwarewise use the same software tools.

DR. GARRA: If the hardware and the software the same as the Advantage Windows, then it sounds like it would have to be the Advantage Windows for the film applications.

MS. SITZLER: The mammo application includes--the non-diagnostic review station includes the 2k by 2.5k monitors which are specifically for mammography.

DR. GARRA: So assuming that you might get a number of users--if you were to market this, you might get a number of users who might use it in a so-called off-label mode where they do start doing soft-copy readings. That is the reason for asking that question, to see what the capabilities of the system were.

The second question I have is regarding image processing. You mentioned about the thickness correction. Are there other image processing parameters that can be performed on the system and, if so--first of all, I will let you answer that one.

MS. SITZLER: Right now, we do the thickness compensation and automatic-contrast determination before presenting the image.

DR. GARRA: Those are the only two currently?


DR. GARRA: No edge enhancement or anything like that?

MS. SITZLER: Not now.

DR. GARRA: What steps did you take to optimize those parameters? How were they optimized, in other words?

MS. SITZLER: How were the parameters optimized?

DR. GARRA: Did somebody say, "Oh; this looks pretty good?" and say, "That is what we are going to use?"

MS. SITZLER: I am hesitating because the whole design process is described in the larger PMA documentation and is part of that whole process where we got feedback from users on the presentation of the images and optimized the parameters based on their feedback.

DR. GARRA: The reason for asking that question is that regardless of how it was selected, the use of improper parameters or of non-optimal parameters could lead to compensation in terms of higher exposure which would be a violation of ALARA. That is the reason for asking.

DR. HENDRICK: These are all post-processing steps.

DR. GARRA: Right. But if you get a poor image, you might be tempted to say, "We have to use more technique." For instance, if you set your brightness or contrast settings to the wrong settings, you might be tempted to compensate by reexposing at a higher dose level.

DR. HENDRICK: I think that is why we need training for people making those determinations on the system.

DR. GARRA: I just want to be sure that you did have a thorough process of optimization and did arrive at what you think are the lowest reasonably achievable doses.

DR. HENDRICK: All these images were required at the same doses as film screen. The images that went into their optimization of the thickness equalization algorithm, they were images acquired in ongoing studies that equalized the dose between film screen and digital.

DR. GARRA: Again, that is a matter of interpretation as to whether--that may not be the lowest reasonably achievable dose.

DR. HENDRICK: No; I am not suggesting that.

DR. GARRA: I will defer to the FDA on whether they want to follow that regulation or not. Do they want to go for lowest reasonably achievable dose or do they want to go for what is currently achievable with film screen. That is a question that maybe should be addressed in the follow-on study at the end if the device is approved.

That's it for me.

DR. DESTOUET: I have one last question before we dismiss the panel. On the user end, if we, indeed, choose to use soft copy and have to compare with the hard-copy image from previous mammograms, does G.E. have any advice to the radiologists how to eliminate the layer? How does one have a t.v. monitor with soft-copy display right next to a viewbox. That is something else that Dr. Lewin also addressed in his article.

It is going to be difficult to interface those viewing conditions.

DR. HENDRICK: One of the things we have learned from the Army study with soft-copy display is that you have to have the soft-copy display monitors in a very dark room because the brightness output isn't as high as the brightness coming through a film to your eye on a viewbox. So the viewing conditions are very important.

There will be the need, if people do soft-copy display, to have a viewing setup where they can compare, say, prior film-screen mammograms to current soft-copy displayed mammograms. But that needs to be worked out at each site that is going to do soft copy. That is a future step, not in the current application.

MR. DONNELLY: I don't think anybody should underestimate the need for quality control regardless of whether it was a digital or a film-screen read. But, to go back to some of the questions, the need, in terms of monitor resolutions--there has been a lot of work done with a lot of radiologists and a lot of evaluation of different kinds of equipment and some parameter settings to optimize basically to the current radiologist's view so that when the hard copy came out, it looked equivalent to what the doctors were used to seeing on the piece of film screen.

But all those kinds of issues, all the way back to the printer settings to is the room the right darkness, have you selected the right monitor, these are all things that we have worked through and established standards for. So when we conducted the trials, these were in place.

Obviously, the right mechanisms have to be in place so that as these machines are deployed in a large number of settings, that that same level of quality control is always in place. We have guidelines, as I said, for printers. We have guidelines on the monitors that must be used. We have guidelines and recommendations on the room, darkness.

But there are things that are outside of what you would expect for a normal film-screen room. A lot of practices that, today, you see for people who are doing high-res imaging in a CTRM modality where you run into a lot of those same kinds of issues, what is the right environment to do reads on a live monitor as opposed to a light panel.

So those have to go with the product to see that it is applied in the proper fashion.

DR. DESTOUET: Any other questions from the panel?

MR. DOYLE: The FDA has three discussion points that they would like to have the panel address. We will put them up on the screen and I will read them.

The first one is, please discuss whether or not the PMA contains sufficient data to conclude that the Senographe 2000D is safe and effective for mammography.

DR. DESTOUET: Is there anyone on the panel?

DR. ROMILLY-HARPER: I think that the PMA does contain enough data to conclude that the Senographe is safe and effective.

DR. DESTOUET: Any dissenting opinions? Any seconds?

DR. GARRA: We don't need a second. This is just discussion.

DR. DESTOUET: We are dealing, basically, with the same piece of equipment that has been on the market for many years except for the image receptor. So it seems as though the Senographe, indeed, is safe and effective and the PMA outlines it as such.

DR. GARRA: I would agree. At least, it seems totally safe and effective to me based on this data.

DR. HARMS: I would agree. I think it is safe and effective and the data provided more than validate that.

DR. MALCOLM: I have no additional comments. I agree with the comments that were made; it is clear it meets the criteria.

MR. DOYLE: I guess we can move on to the second discussion point; please discuss whether the labeling of this device, including the indications for use, is appropriate given the data provided in the PMA application. Are the biases, errors and limitations of the clinical study adequately described in the labeling?

DR. DESTOUET: Dr. Berg, do you have any questions about labeling?

DR. BERG: My only concern was whether we have really established that people will be able to make the right decisions for final diagnosis. Is it a 0, an abnormal or a normal; I think that has been very well established with the data we have been presented. I think there is data that supports the application but it is not necessarily fully included in the application. That is my only comment on that.

DR. HARMS: There was discussion earlier in the public forum about the 510(k) versus the PMA. The demonstration of equivalence to mammography is, on the basis of the PMA, by direct clinical comparison whereas other devices, other digital devices, have not had to do that.

The problem with the PMA is that it doesn't fully demonstrate equivalence to screening mammography because of the patient population. As Wendie mentions, there are some concerns about the diagnostic side as well. Perhaps a better way of approaching this is on the 510(k) mechanism rather than a PMA.

I think the paradigm for this was the Pap screening study where the automated readers of Pap smears are compared with standard readers of Pap smears. I think that may be, in essence, the problem because that focusses on the diagnostic side of things rather than how the data is gathered whereas the data in the case of digital mammograms, the gathering of the data is what we are trying to measure, not the interpretation.

Unfortunately, the PMA focusses on the interpretation. Actually, the interpretation is the biggest variable that we have. So I have some concerns about the mechanism of approval and the FDA guidelines for this, but I would agree that the indications of the device are appropriate. But I am not sure that the PMA actually answers that appropriateness.

DR. GARRA: I would like to make a comment about the labeling. Just looking through the proposed labeling section of the PMA, it is basically a slightly truncated version of the study results and you really have to work to tease out the meaningful differences between this study and a pure screening study. It took us most of the day today to do that.

I would suggest that the labeling needs to be modified with a summary paragraph that surely summarizes the difference between the data collected here and a true screening study and also emphasizes the fact that, although the patient population was a mixture of diagnostic and screening populations, the study, itself, was run in screening mode rather than diagnostic mode so you don't have complete information about either one.

It is enough to be approved, but they have to realize those limitations. That needs to be summarized succinctly at the front end, I think.

DR. DESTOUET: Dan, can the FDA work with the manufacturer to come up with a statement to that effect?

DR. SCHULTZ: Absolutely.

DR. MALCOLM: I agree with the comments that Brian just put forward. Otherwise, I think it fits the question 2 with the modifications that were just suggested.

DR. ROMILLY-HARPER: I would like to make just one comment. Just to the G.E. people, the fact that you do have this type of equipment in other countries, especially the European countries, part of our problem with screening devices in the United States is just the nature of the beast here. Maybe you should make some effort in collecting data from the European countries that will be applicable to describe to this population down the road.

It is very difficult to get true screening data in the United States. It is almost impossible.

DR. DESTOUET: Any other comments about discussion point 2?

MR. DOYLE: Discussion point 3, then; there are issues not fully addressed in the PMA that require a postmarket study to resolve. Will the proposed study resolve these issues?

DR. DESTOUET: Dr. Toledano, have you looked at the proposed study by the manufacturer? Do you have any comments?

DR. TOLEDANO: I think I asked some of my most important questions earlier. I do have one remaining issue with the postmarket study is that I would like to see some plan to describe the variability across mammographers when they are interpreting the digital mammograms.

We know that that is one of our largest problems with the film screen. That is what waylayed the guidance from 1996 and we are still all trying to get back on track. So I would like to know, for digital, is everybody on the same curve as in the Elmore study? Is everybody on different curves as in the Beam study? How much does training affect that, and just what kinds of variability we would see. So I would add something in, some plans to address that.

That is a huge concern for women going to get mammograms, that they don't know, depending on who you go to and what kind of day they are having. It would be nice to be able to quantify that for our population.

DR. HARMS: Unfortunately, that is probably not device-driven. That is probably more radiologist-driven. The concern I have here is that, in screening mammography, we are trying to provide an examination at low cost. If we increase the cost of that examination, then it will no longer be beneficial to society to screen.

I wonder what gain--the further study would be a study of screening. What is the relative gain of screening compared to the cost and the safety issues. We are mandated to make this a safe device and an effectiveness device. The safety, I think, is pretty apparent. The effectiveness for screening is the question here.

It looks like, from the physics data and the data presented so far, that it is likely to be equally effective as standard mammography. But the costs of doing a study to prove that are enormous. That will probably be passed on to the patients, ultimately, and I have a great deal of concern of whether this is worth the effort.

DR. DESTOUET: Any other comments?

DR. GARRA: I also have that concern and, because of that, I sort of hesitate to add an additional study to the postmarket one. The postmarket one that I see proposed here is basically an extension of the screening protocol. If you are labeling it for screening and diagnosis, I think at least a small trial in the diagnostic modes--and the mammographers here would be better able to figure out what needs to be tested, the magnification modes, things like that, would be appropriate as a postmarket study.

I am concerned about the screening component from the cost standpoint as well. I like the alternate proposal where they tag onto some of the Army data as sort of a generic way of increasing number of cases more quickly and more cheaply.

DR. HARMS: The other issue was, again, with the Pap smear paradigm. You could take the same slide and have it read two different ways. But, with this, we actually have to expose the patients twice. That is a significant problem, a significant cost as well as X-ray exposure.

DR. GARRA: I did like Dr. Sacks' comment, though, that it is not like you are gaining nothing by doing the double exposure, as long as it is explained to the patients and they understand that they get an additional exposure risk but they also derive, probably, a significant benefit from the extra exposure. But, again, you still have that cost issue.

DR. DESTOUET: It is going to cost a lot of money and we certainly will have to radiate a lot more women. It seems that the data that has been presented shows that digital mammography is effective in screening. I am not sure how much more data we need before we just give it the go-ahead.

I think Dr. Berg has raised some concerns about working up the diagnostic patient but that could certainly be answered with a much smaller study as opposed to having a full-scale postmarket screening study.

DR. BERG: I would submit that, from the data we have received, in particular, if the study was focussed on the category 3 and 4 lesions, sort of at the threshold and established equivalent performance which, I grant you, is very difficult with readers, but using the same readers to read the same studies, I think that is where the focus would be most effective in really answering the sort of questions that loom in all mammographers minds about the subtle future analysis that could be different between the two modalities.

DR. ROMILLY-HARPER: I tend to agree with both comments that were just made. I think it is easier to prove to an individual and, as a physician, it is easier to know that you are radiating a patient if they are gaining, if it is a significant gain to that individual and their diagnosis, but I have a problem with exposing women in a screening mode who have, really, nothing to gain except for proving that the instrumentation works.

I think we have the data from this study that shows that it is an effective tool for mammography.

DR. HARMS: We are using digital methods quite a bit in our department. We have digital chest units and those were readily approved. I can see a difference in the population as pointed out by the FDA in mammography to other radiographic techniques.

But there are a lot more similarities there than there are differences and it is a lot closer to that than it is to the Pap-smear paradigm. So I would like to encourage more similarity and approval mechanism to other digital media.

DR. GARRA: I have a question about--in looking at the proposal, what access is available to the Army data to increase sample sizes? I have heard that it might be available, that it really isn't available. Can I ask a question to this point on that, because that is probably critical if the data is already there. It makes no sense to repeat it.

DR. HENDRICK: The Army data are there to the extent that there are now 36 cancers and about 7,000 women have been screened. The 36 cancers are based on an analysis that was back when there were just about 5,000 women screened. That information is being written up for publication right now on an interim analysis of the Army data.

The Army data can be made available for further analysis in terms of the number of cases that would be needed for a screening population but any further study of that would be done with new readings, not with the readings done supported by the Army funds. So the cases could be made available. All the cancers in a subset of noncancers could be made available for a further study but all of the readings would have to be redone.

Based on the design the FDA presented and that we, basically agree with, it would require many readers reading those cases to eliminate some of the reader variability issues. So the data could be made available for such a study that would focus on screening-generated cancers. Right now, we know of 36. There may be a few more that have come in since the 5,000 women have been analyzed so we may be up to low 40s in terms of number of cancers at this point.

DR. GARRA: So, given that scenario, then, we don't need to irradiate women again. I think we should explore the possibility--we are talking about basically money to pay readers. I think the possibility should be explored of getting that data and doing a reader study to solidify the numbers that we have sort of large error bars now on, and then a determination, after that point, as to whether any additional study that needs to be made could be made.

But I think it is premature now, without even looking at all the data, to try to do this.

DR. SCHULTZ: Could I make a brief comment. I sort of am going back on my promise not to get into the policy issues, but maybe a couple of small statements might be helpful. We did believe, and we still believe, that there are some major questions with regard--as was discussed in the earlier presentations, not regarding the point estimates but regarding the width of the confidence intervals about those numbers.

And we agree with some of the comments that were made that there are questions, not only with the screening but, also, some questions as far as the diagnostic populations. We also understand that what was presented here was essentially a hybrid and gave us a fair amount of information, obviously information that you have already told us that you think it demonstrates safety and effectiveness.

So, I think, from that standpoint, we have gotten a lot of information out of the study that has been presented here today. But there are some unanswered questions and we would like to do as much as we can to try to get those questions answered, not tomorrow, not the day after, but in some reasonable interval in the postmarket period.

With regard to the cost and with regard to how that data is procured by individual companies, we have discussed with company, as well as other companies, the fact that if there are ways that those costs could be defrayed in a variety of different ways, we would be willing to explore those options with them.

We are also willing to look at ways to cut down on the number of normal studies that need to be multiple read which, I think, is also a large part of the additional cost. There are ways to do these types of studies to get the kind of information with the kind of data and there are ways to do those studies in smart ways.

I think some of those smart ways, and Dr. Toledano can help me here and Dr. Wagner--but there are smart ways to do this to get information that we believe is necessary to ultimately have a better understanding of how these devices are going to function.

But, again, I think there are different ways, number one, to do the studies and, number two, pay for them that would allow these studies to be done over a reasonable length of time.

I hope that answers that question.

DR. HARMS: I have another concern and that is that this is rapidly evolving technology with a lot of new innovation. My familiarity with the PMA is that it is a relatively rigid process. Is there a mechanism here for incorporating new innovations that would be beneficial to patients without locking the companies into some rigid mechanism?

DR. SCHULTZ: I think you are right. Let me not use the word "rigid" so much as a more involved process. We recognize that. That is, again, something that we have heard loud and clear and that we understand. Depending on the changes that are made, we have a number of different mechanisms within the PMA process which allow for minor changes to be made with relatively minor levels of scrutiny and major changes requiring larger levels of scrutiny.

So, for instance, an addition of soft copy to hard copy which, I think, we would all consider a fairly substantial change, would require that the sponsor come in with a supplement which showed the fact that the hard copy and the soft copy were equivalent.

Some of the changes less apt to directly affect the safety and effectiveness of the product could be made with less complete, less burdensome, if you will, types of submissions. We are going to look very hard at that because we do understand that this technology is not going to be static. It is going to evolve over time.

In fact, most of the technologies that we are approving today fall into that category. There are very few technologies that we approve through 510(k) or PMA and, in fact, the most cutting-edge technologies are the ones that go through PMA and those are the very technologies that we know are going to evolve rapidly over time.

So I think the new regulations, some of the new internal changes that we have made, do recognize that and provide us with a number of different ways of evaluating these different changes depending on the magnitude of those changes.

DR. SMATHERS: I am troubled by that last statement about soft copy being a major change because I think the true benefit to the patient is going to come when digital radiography comes out with soft copy and all the flexibility that it offers.

I would relate to Dr. Harms' comments in that you have digital radiography and chest films right now and you have soft copy there. There have been comparisons of soft copy and hard copy there. I really don't, in my view, see that as being a major hurdle to cross in mammography.

So I would encourage the FDA to move soft copy along as fast as possible. My gut feeling is that I really wouldn't release it until you had soft copy because I think that is the true benefit of the system. It is marginally the same right now, the way you are going to release it. Why incur the costs on the medical-care system if you have something that is just as good but not better?

The "better" is going to come in the ability to manipulate the soft copy and have fewer recalls because you can do that manipulation.

DR. DESTOUET: There is actually existing soft-copy display of digital images now with stereotactic biopsy machines so I am not sure why that should be a hurdle at all. We are already accustomed to using soft-copy display.

DR. GARRA: I would like to suggest that I think that the study that is proposed here is looking backwards and I agree with Dr. Smathers and Dr. Harms that we need to look forward. If there is going to be a postmarket approval study, you might as well go ahead and just do it with soft-copy and compare it. You will get the hard-copy data anyway but it will also give you the soft-copy information and you will save a step, at least.

If you are going to incur any costs, then I think it makes no sense to stay on hard copy. Go to soft copy and do them both at the same time.

DR. SCHULTZ: Does the company want to comment? If I understood your proposal, the idea of incorporating soft copy into the postmarket study is already being looked at.

MR. DONNELLY: I think you are right, Dan. The consideration that we have in terms of a postmarket study would be--to Dr. Smathers' turn, would be to very quickly get soft copy included in this PMA, be able to do the PMA as we stand today but very quickly do an amendment to include soft copy as well and for whatever postmarket studies, be it those that we have proposed or modifications, be conducted in a soft-copy environment. We agree with that 100 percent.

DR. GARRA: I wouldn't even spend a dollar on postmarket studies that don't include soft copy. I would wait until you got it and then just do it all at once and save the money.

DR. DESTOUET: Are there any other comments?

DR. GARRA: We have finished with the discussion and now we are ready to open the second half hour of open public hearing. You are reminded that the same identification process--in other words, your name, affiliation, financial disclosure information and a five-minute maximum time limit still apply.

There is one individual that we know would like to speak. If there are any other individuals at this time, would you please raise your hands or please identify yourself to Bob Doyle.

The first speaker is Dr. Earl Steinberg of Covans.

1   2   3   4   5   6   7

The database is protected by copyright © 2016
send message

    Main page