How to deploy CSPP SDR in a container?

VIIRS Sensor Data Record
Post Reply
adybbroe
Posts: 2
Joined: Thu Mar 03, 2022 10:21 am

How to deploy CSPP SDR in a container?

Post by adybbroe »

Hi,

We would like to be able to deploy the CSPP SDR processor in a container, and wonder if you have any progress on presenting a description/manual on how to that?

In a previous post a couple of years back:
viewtopic.php?f=38&t=57507&hilit=container

We see this statement:
"We are working on a document describing how to deploy CSPP SDR Version 3.2 in a container. An announcement will be sent when it is ready,"

-Adam
scottm

Re: How to deploy CSPP SDR in a container?

Post by scottm »

Hi Adam,

There are several container technologies and multiple ways that an SDR installation could be implemented. In order to assist you in an efficient manor it would help if we knew your deployment strategy. Have you thought about where you are going to deploy the container? How are resources distributed in the container environment? Will this be deployed on a single machine with a signal disk or across multiple machine with many instances? Do you have a specific container technology in mind?

Let me know
adybbroe
Posts: 2
Joined: Thu Mar 03, 2022 10:21 am

Re: How to deploy CSPP SDR in a container?

Post by adybbroe »

Hi Scott!

Sorry for the delayed answer. Here some input from us. In Norway, Finland and Sweden we work in parallel and together, with a common goal of sharing some of our satellite data processing in the future. Container solutions will probably be one means of achieving this, so it should be possible to build a container at one institute and shared it to the others. The technology is fairly new for us, and I am yet a naive beginner in this field, so please forgive me.

At SMHI we have Tanzu. It is fairly new, and we have yet no production set up in that environment. We cannot deploy externally created Docker containers, but will rely on creating the Docker image ourselves.
FMI (Finland) is leaning towards OpenShift clusters, but it isn't widely available yet. There are things in Docker, and there's also some Podman installations.
From my Norwegian colleague, Trygve:
"I have been working with docker containers, so I assumed docker containers. But singularity can also work. For the running environment, I don't think running in multiple machines with many instances (like Kubernetes as I understand it or other similar environments). But rather on a single instance. I might use eg. docker-compose for the setup. Another possibility is to create a singularity container from eg. a docker container, and then run the singularity container on our gridengine environment. The container then contains the software needed for the processing, and then the needed data input/output is mounted/binded into the container at run time. The challenge will then be to keep the luts and other needed data for the processing updated."

I think what Trygve says also applies to us at SMHI. And I think it goes for all of us, that we will need to build our containers ourselves, I don't think our IT departments like containers delivered from outside, due to security issues.

Hope that helps you enough to eventually provide us some instructions how we could make a test build with CSPP?
-Adam
Post Reply