AFRL seeks to find a partner with the capabilities to predict the internal structural configuration of buildings by only using external satellite photographs of the building taken from various angles.

Overview

The use of machine learning techniques to enable modeling of complex problems is of significant interest to the Department of Defense (DoD). One goal that could benefit from machine learning is predicting the structural response of buildings, and a key input to this analysis is a good understanding of the building structure and the structural detailing. Often the Department of Defense does not have access to blue-prints or even information on internal layout of a building, but must nonetheless estimate the internal structure before doing an analysis of the structural response of the building due to various loads. The current approach used in developing a target structure for a structural response analysis is for a skilled person sit down and use specialized software to make representative models of buildings and other structures based on standard construction templates and structural design rules of thumb. This is a time consuming process that the Department of Defense would like to be largely automated.

  • What: AFRL Machine Learning Prediction of Internal Building Structures Challenge

  • When: September 1 - November 19, 2021. OCT 15 - Whitepaper Submission Deadline); OCT 29 - Whitepaper selections announced; NOV 12 - Pitch Day; and NOV 19 - Phase 1 of the $1M award will be distributed.

  • Where: Virtual

  • Who: This competition is restricted to participation from personnel classified as “US Persons” (US Citizens or Permanent Residents).

  • Why: The goal of this challenge is to predict the internal configuration of structures such as buildings by only using external satellite photographs of the building taken from various angles.

The Challenge

The goal of this challenge is to predict the internal configuration of structures such as buildings by only using external satellite photographs of the building taken from various angles. Using the shape, size, and external characteristics of the building (such as the number and spacing of windows), the desired methodology should be able to produce a model of the building to include basic structure system (frame vs. load-bearing systems), the construction materials and likely material properties, structural detailing, and any other feature responsible for the overall structural strength of the building. A machine learning model trained with a sufficient database of known buildings, along with constraints to represent current construction processes, could potentially yield a machine learning model that can use a minimum of external visual clues and then automatically produce a digital file of the most likely structural design of the building. The prediction of the number of configurations of non-load bearing partition walls would also be a benefit and should be regarded as an additional secondary challenge.

The desired methodology should be able to produce a model of the building to include basic structure system (frame vs. load-bearing systems), the construction materials and likely material properties, structural detailing, and any other feature responsible for the overall structural strength of the building.

Solution Requirements Background and Description

The solution of this challenge shall use publicly available machine learning tools, such as TensorFlow, and publicly available tools and languages, such as Anaconda and Python. These tools are only given as examples and are not requirements. However, the desire is to use software tools and languages available to a wide variety of organizations both inside and outside the Department of Defense. The intent of AFRL is to use the results of this challenge as a building block for later effort and using open source tools will enable work to be transitioned to later efforts.

The inputs to this challenge will be a dataset of external satellite photos of common buildings and other structures, and this dataset will be compiled as part of the challenge. The DoD interest is in photos of buildings taken with very high resolution satellites, although the team may examine using other aerial photos and modify them to make them the same size and resolution as one might expect from very high resolution satellites. The training and validation sets will have to be of buildings for which the team can obtain details about the structural design.

The output of this challenge is the methodology used to solve this challenge as well as the final verified and validated machine learning algorithm, along with an analysis of the validity and appropriateness of using machine learning as an approach to solving this problem. A critique of the Solver’s approach is highly desired so AFRL can plan later efforts based on the lessons learned of this challenge. AFRL would like to have access to the training data set, the full methodology (codes, input decks, scripts, etc.) to modify and reproduce this work, as well as the data files of the estimated structural design of the buildings used to test the final machine learning algorithms.

  • The first phase of this challenge requires a written response (white paper of no more than 4-6 pages) to describe the basic approach, the technical risks and mitigation strategies, and the submitter’s relevant expertise. The proposed solution should be scoped for a 12-24 month technical effort, clearly state the deliverables (e.g. code, scripts, images, models, etc.), and identify/define solution constraints to the clearest extent possible.

  • The second phase will be to develop and demonstrate the proposed approach that meet the requirements of the challenge. This includes building a machine learning approach to solve this challenge along with identifying the image data set for training.

  • The third phase will be to demonstrate the success of the proposed approach by using the compiled image dataset and training the network, which will then be tested with images not used for training. The government would like the option to then submit new pictures of buildings unknown to the Solvers to see how well the final machine learning model predicts the structure of a known building.

Partners