NIKITAS GAVOGIANNIS

From point clouds and photogrammetry towards Historic BIM and Digital Twins:
A methodology for semantic enrichment of cultural heritage monuments

1. Introduction – The need to combine new technologies for a holistic documentation of Cultural
Heritage Monuments

Thematic area. The accurate and detailed documentation of cultural heritage sites is a global goal set by international organizations and research initiatives. Especially in Greece, there are monuments that have only manually been documented without any use of digital media, even though the
means have been existing for decades. At the same time, many different technologies are currently aiming at interoperability, which creates the circumstances for the design of a workflow for highly detailed and accurate documentation and presentation of monuments.

Research goal. The research proposal aims to define a step-by-step workflow, starting at capturing the monument as point clouds, which will be translated to HBIM objects (Historic Building Information Modelling) through AI and ML algorithms. The resulting model is then imported in game engines,
along with GIS data. The goal is to create a smooth transition from one platform to the other, without data or meta-data loss and to visualize the final layered result with different user rights to multiple stakeholders.

Research Hypothesis
The first hypothesis is to produce through the proposed workflow a virtual twin of the monument that will be able to be presented, studied and analysed. The second hypothesis is that the final result will easily allow the creation of advanced CMPs (Conservation Management Plans).

2. Research Methodology – Workflow analysis
The workflow is based on the following steps:
a. Use of digital documentation media (laser scanners on drones) to produce the original point clouds. Except for creating point clouds, scanners also capture in detail the materiality of the monument, as well as its pathology.
b. Use of BIM procedures to convert the point clouds to informative objects. The recognition of specific architectural elements can happen through AI and ML algorithms. The final result is a complete HBIM model which, apart from the geometry itself, contains also meta-data.
c. Use of Geographic Information Systems to add infrastructure and IoT real time databases and be able to analyse the monument on an urban scale.
d. Import the above databases to Game Engine Environment. The monument can then be presented in photoreal mode, through VR and AR. The main objective though is to create stand-alone platforms, where different stakeholders can have access to the data of the
monument with different user rights.

Expected outcome
The workflow needs to be tested in selected case studies of cultural heritage monuments, since each one is characterized by its own specific conditions and complexity. Greek monuments in particular most often go quite back in time. Every period would bring unique architectural characteristics, typologies and materials. Those parameters should be carefully documented and presented in order to get a good understanding of the monument. Moreover,
many monuments often belong or are managed by more than one agency, whether public or private. This brings extra complexity in the creation of CMPs.
Therefore, the expected outcome is the design of a new road-map, adjusted on the specific characteristics and needs of the cultural monument, which will convert in the best possible way the data to a complete digital twin.

3. Research challenges – Points of interest – Game Engines
The research aims to focus on the following points within the workflow:
a. Conversion of points clouds to BIM objects through AI and ML algorithms.
b. Use of 3d GIS protocols (CityGML).
c. Import of the above into one layered model in game engine environments (Unity 3D or Unreal).
d. Creation of stand-alone applications with different user rights.
While a) has already been the focus of many research initiatives and b) is still complicated since GIS have only recently invested in interoperability with 3d geometries, c) is used more often for visualization and less for management or planning purposes.
Therefore, although a) and b) will be thoroughly analysed and taken into account on the development of the workflow, c) and d) will be the main focus. Final aim of the proposed research
is the creation of a digital twin of the monument and the procedure that leads there.