Data visualization of ship movements in the port of Hamburg
How the Hamburg Vessel Coordination Center uses data analytics to gain new insights into ship movements in the Port of Hamburg
30,000 to 40,000 different ships and 400 messages per second are received worldwide via the AIS system for tracking ships. For the Hamburg Vessel Coordination Center (HVCC), we analyzed and visualized a section of the AIS data for ship movements in the Port of Hamburg in more detail. Together with the HVCC, we developed a proof of concept that showed new potential for better coordination of ships by visualizing the data.
The Hamburg Vessel Coordination Center (HVCC) is the central coordination point for shipping traffic in the Port of Hamburg. It controls the movements of container and inland vessels and gives instructions for their onward journey. The HVCC wanted to get a better picture of the traffic in the Port of Hamburg and find out how to optimize the movements. As data specialists, we analyzed the status quo, developed a cloud-based data analytics concept from data preparation to analysis, and visualized the results in looker dashboards.
Ship data in Google Cloud
The basis for our proof of concept was the available real-time data. The advantage in shipping: The so-called AIS system (Automatic Identification System) is mandatory and sends data on the position speed, direction and identification of the ships at regular intervals. But a closer look at the data situation presented us with some challenges:
- The available data was patchy or of insufficient quality.
- A few vessels are exempt from mandatory AIS.
- The HVCC did not have consistent historical data.
As a first step, we then developed a holistic approach to data acquisition and preparation, as well as a suitable database structure for storage. We stored the data stream of the AIS community and built a database of historical data. In addition, we increased the volume and quality of streaming data: Recommendations for better data collection closed gaps in AIS data and smaller vessels should be equipped with GPS transmitters.
To make this pool of data accessible for visualization and analysis, we developed a unified database in the form of a BigQuery data warehouse. Customer applications can access the stored data via corresponding APIs.
Monitoring with customized dashboards
For the optimal coordination of shipping traffic in the Port of Hamburg, the HVCC faced the same recurring questions:
- How long do ships stay in the port?
- Where do the ships stay in the port?
- Can unique locations (e.g. terminals) be identified?
- Which stations do the ships call at and in which order?
- Can waiting times before docking be identified?
- Are there any abnormalities in speed?
In order to not only get a better picture of the traffic in the port, but also to be able to optimize the planning and loading of the ships, the HVCC wanted an easily accessible solution. Based on Looker, we developed several dashboards as a proof of concept that show ship movements on interactive maps. The dashboards allow both the visualization of historical data and the monitoring of current vessel traffic in real time.
On the one hand, this gives the HVCC access to all previously aggregated vessel data and allows them to intuitively explore it over time on a map. Monitoring ships is now easier as well. A real-time view provides a look at the current situation in the Port of Hamburg and detailed information on all ships. In addition, a further dashboard allows the catchment areas up to the North Sea to be monitored.
Potential for optimization thanks to machine learning
Once the foundation of data collection, processing and visualization in Google Cloud is in place, real-time monitoring and historical analysis are easy. If you follow the data journey further, a prototype like the HVCC can be easily expanded with Google's machine learning technologies. Only then will data-based forecasts and recommendations for action become possible.
In the case of Hamburg's shipping traffic, for example, it would be possible to forecast the arrival time of individual ships and predict future traffic density based on various factors:
- Timetables
- Historical journey profiles
- Environmental data (weather, water levels)
- Current speeds
Such forecasts hold enormous potential for optimization. The HVCC could, for example, improve the coordination of inland vessels by optimizing transit and clearance times and responding more effectively to schedule deviations. There would also be opportunities for optimization in the use of resources: improved plannability could reduce waiting times as well as standby and operating costs.
