In 2010, NASA was tasked to collect all of the Agency’s technology development information into a single repository to facilitate analysis, decision-support, and communication. Because ARES Corporation is known for our demonstrated capabilities with integration, enterprise strategy, portfolio analysis, and software development, NASA consulted with our subject matter experts to help with the design of an Agency Technology Portfolio Management System, commonly known as TechPort. ARES conducted a trade study using our risk-based decision-making approach to analyze solution alternatives and formulate a baseline for the system requirements. Through knowledge sharing forums, round-robin discussions, and whiteboard exercises, our team collected stakeholder inputs, and evaluated software and hardware options across cost, schedule, and technical risks. In less than a year we securely implemented and deployed the first version of TechPort. ARES has successfully operated TechPort at NASA for over a decade.
“…the ARES Team consistently continues to come up with creative solutions for difficult problems, they are also extremely effective in listening to contributing stakeholders – creating an environment encouraging innovation and efficiencies.” – NASA TechPort Customer
NASA tracks over 16,000 technology projects across the Agency in TechPort, from propulsion systems to robotics to ground systems and software. TechPort is a dynamic system, allowing individuals involved in the development of technology to update the system real-time with a secure, user-friendly form. Frequently, an organization has most of the information needed to conduct portfolio analysis stored in disparate systems and spreadsheets. TechPort integrates these data sources using an application programming interface (API) connector and semi-automated bulk-import alternatives. TechPort contains the ability to create hundreds of different types of reports in various formats, such as Number of Projects by Taxonomy Element, Investment by Technology Maturity, Locations where Work is Performed, and many more. TechPort features a simple keyword search powered by semantic technologies and industry-specify synonyms, as well as a faceted advanced search allowing users to search by multiple combinations of project record fields.
TechPort is a scalable system designed to track massive amounts of data. TechPort was NASA’s first enterprise cloud-based system hosted in Amazon Web Services GovCloud, and the ARES Team served as pathfinders for the Agency’s cloud compute initiatives. TechPort data are highly dynamic, changing on a daily basis with an average growth rate of about 58% per year as new technology development is funded.
Challenge: Technology Classification
Accurate classification of technology projects provides the basis for search, data extraction, and infusion identification. As such, NASA utilizes a technology taxonomy to provide a structured means of organizing technologies. In 2020, NASA revised the taxonomy, expanding the number of elements and providing new categories for emerging capabilities. ARES was tasked with applying the new NASA Technology Taxonomy to TechPort data records. Requiring over 10,000 individual project managers to familiarize themselves with 387 classification possibilities would require cost-prohibitive training and verification, estimated in thousands of hours. Instead of a manual approach, the team recommended the development of a machine learning model that would automatically classify the 16,000 existing projects, and provide a future-proof solution that recommends taxonomy classifications to managers when new records are added to TechPort.
The Solution: Machine Learning
To solve these challenges, we designed, built, and deployed a system called T-Rex, or the NASA Technology Taxonomy Recommender System. The contributions of T-Rex is notable across multiple areas from model selection to optimization and deployment. Our recommender is integrated into TechPort in a novel, effective manner and other groups at NASA are now using T-Rex to programmatically classify technology through an exposed API. The system has since been used in other projects to train and build recommenders, such as the NASA Technology Target Destination system. We found our model achieves over 96% accuracy in k-fold training, and over 97.5% accuracy in measured user updates.
T-Rex is an amalgamation of classifier models, custom models, and voting optimization and pruning methods for efficient, accurate, deployed models. The system automatically optimizes, selects, and combines multiple individual models. It uses prior classification information only when accurate to suggest a subset of classes as recommendations for other models. Our training algorithm optimizes weighting from individual models from a matrix of individual model outputs to reduce training time.
T-Rex has been deployed now for over two years and has recommended over 40,000 classifications. The methodology has since been extended to other applications within the Agency, including the identification of target technology destinations and gap analysis.
Read more about T-Rex in the 2022 whitepaper: https://ntrs.nasa.gov/citations/20220016471
TechPort can be found at https://techport.nasa.gov.