EDICT

This paper introduces ”Smartsense”, an innovative technological framework combining modern visualization, Internet of Things (IoT), and Augmented Reality (AR) technologies to support tourism and research in the Danube Delta. It features an interactive map showcasing points of interest, an AR component providing immersive experiences, and IoT-enabled weather stations for real-time environmental monitoring. This paper delves into the development, implementation, and evaluation of Smartsense, emphasizing its potential to revolutionize exploration and study in the Danube Delta through advanced technology, ultimately fostering sustainable tourism, research efficiency, and a deeper understanding of this unique ecosystem.

Keywords: modern visualization, IoT, augmented reality, interactive map, tourism, research

1. Introduction

According to the latest statistical data, at the end of 2022 there were 6.59 billion smartphone users worldwide, with a potential increase to 7.27 billion by the end of 2025. In other words, more than 83% of the planet’s population will use a smartphone by 2025 (Taylor, 2023). We are therefore used to using the smartphone for almost everything we need.

Innovative visual recognition technologies are used in the context of artificially created environments based on high-performance computing systems, presented to users through a variety of display equipment. In such a scenario, the environment can be expanded with additional information, both visual and acoustic. This additional information is represented by bi/three-dimensional images added to the environment, which can be viewed by people in the respective context using common equipment such as tablets or mobile phones.

Also, in scenarios based on innovative visualization technologies, objects considered points of interest, respectively those of context, can coexist in the same space. Thus, these technologies can help develop new ways to experience interactions or various physical environments using a suite of digitized visual sensing techniques.

The possibility of enhancing different real contexts by complex visual methods reduces the need for elaborate and expensive sets of external sensors that translate real environments into digital representations. This can be achieved by attaching sensing technologies such as Intel RealSense cameras to headphones and other smart and connected devices. Initially, the IT community used this technology in specific situations that were not intended for the public, an example being the helmets of fighter pilots.

Due to the widespread dissemination of high-performance computing systems, as well as due to the unprecedented development of mobile devices, complex graphics have become available to any user who owns a smartphone or tablet. The interest for this new visualization technology was observed especially in the cultural area, where cultural institutes, museums, and archaeological sites had the opportunity to offer visitors an immersive, interactive cultural experience, with a much better yield in the transfer of knowledge. Due to the widespread use of smartphones these new technologies can now be also applied to the tourism industry.

The Danube Delta, a UNESCO World Heritage Site, is one of Europe’s most significant and unique natural habitats. The region boasts a rich biodiversity and attracts thousands of tourists and researchers annually. However, the delicate ecosystem faces environmental challenges and requires efficient management to ensure its long-term preservation. Smartsense aims to address these issues by leveraging modern visualization and IoT technologies to enhance tourism experiences and promote research activities in the Danube Delta.

2. Materials and methods

In this context, Smartsense proposes the use  of an innovative, scalable technological platform, consisting of a suite of software applications based on modern visualization, IOT components, data analysis algorithms, and modern technologies of transmission, in order to support and improve tourism and research activities in tourist areas and reservations. The technological framework will be customized for the Danube Delta biosphere, but there will be the possibility of adaptation for any tourist area or natural area.

The proposed technological solution is built around the following elements: a multilayer interactive map, available for both mobile and desktop devices, an augmented reality component, a data analysis collection component, a data analysis module, a component for visualisation of the results obtained as well as the collected data and a data sharing module.

SmartSense uses the specific technologies of augmented reality to create an interactive tourist platform for exploring and promoting the Danube Delta area.

The SmartSense platform is centered around the notion of „ecological tourism experience”, offering the tourist the opportunity to interact, through augmented reality technologies, with the environment without having a negative impact on it. Thus, the user can get involved in activities of exploration, research, discovery of certain elements specific to the Danube Delta. Using the smartphone’s interactive map and GPS location technology, he can access the experiences closest to him.

The platform offers the user access to educational, gastronomic, architectural, entertainment and sports experiences specific to the Danube Delta area. The experiences allow interaction with elements of flora and fauna, local customs, and archaeological sites, to which access is sometimes difficult in the Danube Delta area. An experience can have several associated points of interest, thus making it possible to define route-type experiences where the user can explore a wider area of the Danube Delta area.

The local community can contribute to the variety of experiences by being able to define and customize new experiences and points of interest.

At the same time, the SmartSense application also offers a component for monitoring air and water quality parameters and notifying the user when they exceed the permissible limits. The user can view the most recent sensor readings and receive alerts if the recorded values surpass a specific threshold over an extended period.

3. Results and discussions

The software architecture has been designed taking into account criteria of performance, interoperability, portability, reliability and compatibility of various components. Thus, the Smartsense platform, from the perspective of the functional components, is presented in Figure 1. The main components are: the interactive map, the AR component, the IoT component and the backend component – which allows the easy management, by the platform administrators, of the content found in the public interface and the mobile apps.

4.1 Smartsense IoT component

The IoT component is built on a three-tier architecture consisting of sensing, communication, and application layers. The sensing layer comprises IoT devices strategically placed throughout the Danube Delta to collect real-time data on various environmental parameters. The communication layer facilitates the transmission of data from sensors to cloud-based servers. Lastly, the application layer processes and visualizes the collected data, providing accessible information to tourists and researchers.

The Smartsense framework employs a range of IoT sensors, including temperature, humidity, atmospheric pressure, solar radiation, water and air quality sensors. These sensors generate continuous streams of data, which are collected and stored on the servers and are displayed to the users or processed for classification and predictions.

Smartsense integrates the collected data into a user-friendly, interactive platform accessible via mobile devices and web applications. Visitors can access real-time information on weather conditions, water and air quality, allowing them to make informed decisions about their activities in the Danube Delta.

Smartsense also provides researchers with a wealth of data, enabling in-depth analysis of the Danube Delta’s ecosystem. The platform allows researchers to remotely monitor environmental conditions, streamlining data collection and reducing the need for on-site visits. Furthermore, Smartsense facilitates collaboration among researchers by providing a centralized platform for data sharing and analysis.

The section for viewing and analyzing data collected from sensors  (Figure 2) allows filtering according to station and sensor, downloading raw data in json/csv format as well as viewing data in the form of interactive graphs over the desired period.

Also, the visualisation module displays a composite air quality index (AQI) calculated based on the data collected from the sensors. The Chartjs open-source library is used to visualize the data collected from the sensors in the form of graphs.

The IoT component for acquiring and transferring data from weather stations uses the MQTT (Message Queue Telemetry Transport) standard for communication between the server („message broker”) and several clients, on the „publish and subscribe” model . MQTT does not encode data before transmitting it, it is an Application layer protocol.

The component takes the data measured by the sensors, processes it and saves it in the database (PostgreSQL) from where it can be accessed by the client (the public web interface and the mobile application) through a RESTfull API.

4.2 Interactive map component

The interactive map (Figure 3) allows the display of points of interest, experiences and weather stations, based on the user’s location, filtering by interest categories/tags as well as zoom in/out, pan, search, display by scale, etc. functionality.

Points of interest and experiences are displayed dynamically, depending on the chosen scale – for increased performance – and allows filtering by tags and categories. It is also allowed to search by keywords of an experience or point of interest. The user can save his favourite points of interest for easy access later from his personal account.

Open-source OpenStreetMap technology was chosen for building the map, taking into account the readiness for a high number of accesses.

The map communicates with the server through the secure REST API and allows viewing the changes made by the administrator or users in real time.

4.3 IOS and Android mobile apps

The mobile application will be available for both IOS and Android. It allows viewing points of interest (POI) and experiences on the interactive multi-layered map, displaying POI details, viewing and interacting with augmented and non-augmented experiences in the area, displaying details of the chosen experience and accessing experiences based on visual recognition.

The multi-layer interactive map is the main component of the mobile application. It has been designed taking into account the technological limitations that may occur in a low-connectivity environment, and has the following basic functionalities: acquisition and display of the user’s current position, the possibility to interact with the map (zoom in/out, pan, re-centring), the possibility of establishing the display type (vector or raster), displaying points of interest or experiences on various layers (filtering POIs/experiences according to category, tags, etc.), display compass and current position of mobile device in GPS coordinates.

The innovative character of the application lies in the experiences that use augmented reality and visual recognition technologies. They allow an increased degree of interactivity between the user and the mobile application.

The user thus has access to various AR (Figure 4) and non-AR experiences such as virtual museum, birdwatching etc., and is allowed to review the experience he took part in.

Augmented experiences within the application use the technical principle of recognizing objects or geographic coordinates (GPS) that will be overdefined, in real time, with three-dimensional graphics, informational content in the form of pictures, text, audio, video and will be exposed to the user through screens of mobile devices.

The chosen development environment for this component of the SmartSense package is Unity 3D. In general, Unity 3D is used for the development of games but also multimedia applications, virtual and augmented reality applications. Another factor that led to the choice of this solution is the quick possibility of exporting the mobile application for the main operating systems running on most current mobile devices, namely Android and iOS.

4. Conclusions

The use of technology has been one of the proven effective ways to uplift many industries, including the tourism industry. As the technology becomes more accessible and offers high flexibility, its adoption is in high demand in the industry.

The SmartSense application, through the innovative technologies it uses, can become a useful tool for local communities, tourists and researchers who want to carry out tourism, promotional, educational or research activities.

The SmartSense technological framework is customized for the Danube Delta biosphere, but it can be adapted for any tourist area or natural area of interest.

References

Taylor, P. (2023, March 30). Mobile network subscriptions worldwide 2028. Statista. Retrieved April 19, 2023, from statista.com/statistics/330695/number-of-smartphone-users-worldwide/

Stanciu, A.; Cristea, D.; Ivanov, M.; Nazare, A.-K. (2023). Realitate augmentată și geolocalizare în educație: Aplicația Smartsense. In Revista Profesorului 5/2023. https://revistaprofesorului.ro/realitate-augmentata-si-geolocalizare-in-educatie-aplicatia-smartsense/

Stanciu, A.; Cristea, D.; Ivanov, M.; Nazare, A.-K. (2023). SMARTSENSE, prototip de sistem software pentru activități educaționale și de cercetare bazat pe tehnici de realitate augmentată și geolocalizare. In iTeach: Experiențe didactice 5/2023. https://iteach.ro/experientedidactice/smartsense-prototip-de-sistem-software-pentru-activitati-educationale