SmartKai is a digital assistance system that aims to make damage caused by maneuvering in the tight space of a harbor basin a thing of the past. Lidar sensors installed in the harbor capture each ship’s position data, and with the help of the RTMaps software, the data is then prepared for display to the ship’s crew.
SmartKai is an application-oriented research and development project complying with the funding guideline on innovative harbor technology (IHATEC) issued by the German Federal Ministry of Transport and Digital Infrastructure (BMVI), in which several partners are cooperating:
- Niedersachsen Ports GmbH und Co.KG (NPorts) – Germany’s largest port operator and coordinator of the SmartKai project. NPorts makes its harbor infrastructure available for the development of SmartKai. Quays and harbor locks with a high risk of accident are good locations for using the system.
- SICK AG – Is developing a new, more robust lidar sensor for the project, designed specially for a maritime environment. The sensor uses an adapted light wavelength with a longer range.
- Humatects GmbH – Is developing SmartKai’s user interface so that ship crews can easily visualize navigation data using a tablet, augmented reality glasses, or projections.
- OFFIS – Informatics institute at which the harbor’s lidar sensor system for capturing ship positions is being developed.
- eMIR (eMaritime Integrated Reference Platform) – Development platform for maritime applications.
Parking assistants have long been a familiar feature in cars. Now ships will soon have something similar. Maneuvering a ship in the confined space of a harbor basin can be difficult due to adverse currents, bad weather, the increasing size of ships, the growing amount of ship traffic, tight schedules, and so on. The result is repeated accidents that cause damage to both the vessels and the harbor infrastructure – and sometimes personal injuries. The SmartKai project aims to prevent this.
Lidar Sensors in the Harbor Detect Ships
SmartKai is a “parking assistant” for ships. Using lidar sensors mounted on the harbor infrastructure, SmartKai detects the positions of ships during docking and casting off, prepares information from the data, and sends this to the crews and pilots. To gain a precise idea of their ship’s position and maneuver it safely, all the crew on board needs is a tablet, called the portable pilot unit (PPU). Moreover, if there is an accident, SmartKai makes it possible to find out what caused it. Another plus with this system is that no complex, expensive installations are needed on board, because the sensor and computing technology is installed permanently in the harbor.
First Prototype in Wilhelmshaven
The first step was to test the system with no disturbing effects, so Niedersachsen Ports (NPorts) installed it at Hannover Quay in Wilhelmshaven, an almost completely closed harbor area with almost no currents or tides. Three 2-D lidar sensors from project partner SICK were the basis for SmartKai. In addition, environmental sensors (for wind data and visibility) and an AIS receiver were installed (AIS = Automatic Identification System, an internationally standardized radio system for exchanging navigation and other vessel data). Two cameras for visual monitoring were mounted at strategic points on the quayside. All the sensors are connected to two industrial PCs (IPCs) in mobile boxes on the quay. These collect, save, and process the measurement values from the sensors.
For us, RTMaps is the right software for capturing, accurately time-stamping, synchronizing, processing, and forwarding the measurement values from our sensors.
Processing the Measurement Values with RTMaps
For a safety- and time-critical system like SmartKai, reliability and processing speed are essential. The specific challenge is to assign accurate time data to the captured measurement values and synchronize the processing instances, which are spread over a wide area. So the obvious course was to use the RTMaps software (Real-Time Multisensor Applications), which was created for just such scenarios where users have to collect, time-stamp, synchronize, and replay data from different sensors. To handle the challenges posed by the SmartKai project, there is an RTMaps run-time instance on each of the industrial PCs. In addition, all the RTMaps instances are synchronized with one another to ensure that the system is real-time-capable. Another of the project’s main aims is to store all the incoming data permanently to make it available for further development and evaluation of the system. After over a year in operation, an extensive collection of lidar, camera, and AIS and wind data has been collected. RTMaps is also able to replay this data synchronously.
Wide Range of Scenarios Tested Extensively
In September 2020, the system in Wilhelmshaven underwent its first three-day field test with more than 20 scenarios. These were oriented to the regulations issued by the International Maritime Organization (IMO), as well as to requirements stated by pilots. The research boat Josephine (belonging to OFFIS e.V.) and the survey ship Argus (NPorts) were used for the test. The sensor measurement values were saved via RTMaps and appropriately annotated. Data on the start and end times of the scenarios, plus details of any issues or events such as passing ships, were saved in the RTMaps EventMarker format so that the data could be used for the further development of the system. Following that, the system continued in operation through winter 2020/2021 and summer 2021. Docking maneuvers under harsh weather conditions are of particular interest because they can show the effect of the weather on the lidar measurement values. Throughout the entire test phase, data was continuously captured, time-stamped, and saved via RTMaps.
Tests to Be Completed in 2021
The project is designed to run for three years (project end is November 2022), and the next step will be to install another system in a different location, this time Europa Quay in Cuxhaven. The focus there will initially be on the effects of currents and the weather, and how they might make docking difficult. Seven more lidar sensors have been added to the system for this in order to cover the 300-meter-long quayside. The system will be completely tested and evaluated for the first time in the test campaign planned for the end of 2021 in Cuxhaven.
About the author:
M.Sc. Jan Mentjes, research associate in the institute’s transportation division, OFFIS e.V. – Institute for Information Technology, Germany.