Robert McCarthy
The results from a study, funded by the Center for Open Science and ScienceExchange, that aimed to evaluate the replicability of published results in cancer biology, have brought to light a significant issue in life science research.(1) The study, spanning over eight years, attempted to replicate results from 193 experiments described in 53 published papers in cancer biology. The findings were startling, revealing a ‘dirty little secret’ of our field: in most cases, the results from peer reviewed, published research, could not be repeated by their peers. Let’s delve into the details:
- Sharing reagents
- 70% of experiments required asking for key reagents
- 69% of experiments needing a key reagent were shared by the investigator (48.3% of total)
- Authors response to address issues / answer questions
- 41% very helpful
- 32% not helpful or unresponsive
- 0% of the experimental protocols were completely described
- 67% required modifications to perform the experiment
- 41% of modifications completely implemented (27.5% of total)
The conclusion from the effort above and those published earlier (2-4) indicate that most results published in the life science research literature cannot be repeated.
The lack of reproducibility in life science research, as highlighted by this study, may come as a shock to some. However, it’s not surprising to those familiar with the nature of biological research and the competition for limited funding. This does not imply the publication of fraudulent science but underscores the complexity of these experimental systems. In most cases, biological methods cannot be explicitly defined, and a great degree of tacit (i.e., unwritten) knowledge is critical to the success of the research. This is why many experienced labs resort to methods like sending technicians to other labs to learn new procedures or hiring personnel from these labs to teach them the method.
The best way to address this conundrum is to use defined materials and processes whenever possible. For cell isolators, the first place to look is at the source of your collagenase enzymes. Collagenase is a critical component for any enzyme-mediated cell isolation procedure.
In a recent blog post, I described three paradigms for using collagenase in cell isolation procedures. The first paradigm (Everything is good) uses traditional crude or enriched collagenase for cell isolation. These products are minimally processed, Clostridium histolyticum fermentation supernatants. Typically, the protein in the culture supernatant is precipitated with saturated ammonium sulfate, resuspended in a smaller volume, dialyzed, lyophilized, with the appropriate amount of lyophilized protein delivered to a bottle sold to customers.
Crude and enriched collagenase products contain about 3-8% and 15-25% collagenase per g dry weight, respectively. Moreover, the C. histolyticum supernatants contain variable amounts of protease that can degrade collagenase, affecting its ability to degrade/cut native collagen.
Each lot of traditional collagenase is unique. Many laboratories pre-qualify new lots of traditional collagenase before use to ensure that the results obtained with the new lot are similar to those obtained with their current lot of collagenase. The unique biochemical characteristics of each lot of collagenase are caused by several factors including:
- variable secretion of collagenase or protease enzymes during each fermentation;
- proteolysis of collagenase enzymes during fermentation or downstream processing that affects their ability to degrade collagen;
- and variable amounts of endotoxin that can activate cells.
Boehringer Mannheim Biochemicals developed the Liberase Purified Enzyme Blend product line in the mid-1990s to overcome this problem.(5) The project’s stimulus was clinical research scientists’ inability to find “good lots” of collagenase needed to generate sufficient human islets for subsequent transplantation to treat adult type 1 diabetic patients. Liberase HI’s development led to the creation of a second paradigm, “Pure is perfect.” Enzyme purification minimized endotoxin contamination and removed the majority of contaminants from crude collagenase.
However, about 10 years after Liberase HI’s product’s launch, lot variability became a problem because of a change in the purification process. This was further compounded when NIH switched collagenase suppliers at the beginning of an NIH-funded clinical trial assessing the utility of human islet transplantation for managing adult type 1 diabetic patients. The three most experienced labs in human islet transplantation had difficulty generating sufficient islets for subsequent transplant when they used collagenase and protease from the new enzyme supplier.(6)
VitaCyte found that the suboptimal islet yields obtained after the switch in collagenase suppliers could be explained by increased proteolytic degradation of the purified collagenase, affecting its ability to degrade native collagen. These results led to the creation of a third collagenase paradigm: “Aligning collagenase function to application”.
A mechanism model for explaining collagenase-protease-mediated cell isolation was developed to explain these results.(6, 7) Here, the primary role of collagenase is to loosen the extracellular matrix (ECM) so that protease-sensitive sites on the ECMs “cell anchoring proteins” are cut, freeing the cells from the tissue. Collagen is critical in providing a protease-resistant superstructure that protects the ECM from proteolytic damage. This fact explains why the quality of collagenase used in cell isolation is critical to success.
Advances in collagenase purification and characterization enabled the development of optimal collagenase-protease enzyme mixtures for cell isolation by reverse engineering “good lots” of collagenase products used for specific cell isolation or using a prospective design of experiment approach. In all these cases, the assumption is that collagenase’s collagen degradation activity (CDA) is in excess. This step ensures that collagen will be readily degraded, enabling the cell isolator to focus on the dose and characteristics of the neutral protease enzymes used in the isolation procedure. If the CDA activity is in excess, then neutral protease enzymes impact the cell yield, viability, and function.
How is the discussion above relevant to the discussion on practicing open science? Adoption of the third paradigm “Aligning collagenase function to application,” emphasizes the need to use a consistent and rigorously characterized product for your application. The use of these products for cell isolation enables users to:
- confidently share their methods with others with the expectation that similar results will be obtained by your peers;
- fewer factors to focus on when troubleshooting poor performance in an isolation procedure;
- and the ability to make improvements to the method.
VitaCyte encourages all users of its products to publish their protocols in Methods journal or on online sites such as Protocols.io, protocolexchange, or bio-protocol. Please contact technical support at VitaCyte if you want more details on VitaCyte’s products.
In addition, specific cell isolation applications on VitaCyte’s website that reference the use of VitaCyte’s products will be highlighted. This enables you to contact the authors of the protocol if you have any additional questions. Also, VitaCyte technical support is always available to answer any questions you have about its products.
VitaCyte, also intends to publish the quality control protocols that VitaCyte uses for characterizing their collagenase and protease enzyme products on Protocols.io.
References
1. Errington TM, Denis A, Perfito N, Iorns E, Nosek BA. Challenges for assessing replicability in preclinical cancer biology. Elife. 2021. Epub 20211207. eng.
2. Begley CG, Ellis LM. Drug development: Raise standards for preclinical cancer research. Nature. 2012; 28:531-3.
3. Collins FS, Tabak LA. Policy: NIH plans to enhance reproducibility. Nature. 2014; 505:612-3.
4. Bishop D, Cantrell D, Johnson P, Kapur S, Macleod M, Savage C, et al., editors. Reproducibility and reliability of biomedical research: improving research practise. The Academy of Medical Sciences, Symposium Report; 2015.
5. Fetterhoff TJ, Cavanagh TJ, Wile KJ, Wright MJ, Dwulet FE, Gill J, et al. Human pancreatic dissociation using a purified enzyme blend. Transplantation Proceedings. 1995;27:3282-3.
6. McCarthy RC, Green ML, Dwulet FE. Evolution of enzyme requirements for human islet isolation. OBM Transplantation. 2018; 2:[1-30 pp.]. Available from: http://www.lidsen.com/journals/transplantation/transplantation-02-04-024.
7. McCarthy RC BA, Green ML, Dwulet FE. Tissue dissociation enzymes for isolating human islets for transplantation: Factors to consider in setting enzyme acceptance criteria. Transplantation. 2011;91:137-45.