Valiton Technology Radar
Stand: Feb. 2022
In unserem Technology Radar haben wir für uns relevante Technologien und Prozesse zusammengefasst und für uns auf Basis unserer täglichen Arbeit bewertet. Dinge, die wir in neuen Projekten möglichst nicht mehr einsetzen möchten (HOLD), sind hier genauso aufgeführt wie Elemente von deren Relevanz wir auch für die Zukunft überzeugt sind (ADOPT).
Languages & Frameworks
- 1. Doctrine
- 2. Eslint
A tool for static code analysis which replaced Jasmine some time ago. We often use it together with Prettier and the Airbnb style guide.
- 3. Gatsby.js
- 4. gensim
- 5. Huggingface Transformers
- 6. Jest
- 7. Pandas
A program library for the Python programming language that provides tools for managing data and analyzing it.
- 8. React
- 9. React Testing Library
- 10. Sass
- 11. Serverless Framework
We use the serverless framework pretty much everywhere we deploy AWS Lambda, for example. We also try to stay as provider independent as possible in the code.
- 12. spacy
- 13. Symfony
- 14. Terraform
Terraform is an Infrastructure as code (IaC) tool that lets you create, modify, and version infrastructures securely and efficiently. We use it extensively in all our cloud projects, and actually deploy any type of infrastructure in the cloud only using terraform.
- 15. Vue.js
- 41. Cube.js
- 42. next.js
- 43. post-css
- 44. PyTorch
- 45. RxJS
- 46. Spark
Enables us to automatically distribute our data processing across multiple machines
- 47. Tailwind CSS
- 70. ARGO Project
Argo is a project which, with the areas of workflow management, eventing and CI/CD, brings with it several aspects that are of interest to us. We would like to take the opportunity to put these to the test and compare them with our established tools (e.g. Airflow).
- 83. Angular
Framework for building modern mobile and desktop applications. Due to the larger community, we decided to focus on React and Vue.js, and therefore put Angular on hold.
- 84. Less
- 16. Apache Airflow
Open Source Workflow-Management-Platform, we use it heavily in most data projects.
- 18. AWS Organizations
We use AWS Organizations to maintain and govern our AWS infrastructure and the one of our customers. It enables us to simplify plenty of everyday tasks and supports us in applying our security standards.
- 19. dbt
Command line tool that enables data analysts and engineers to transform data in their warehouse more effectively. We use it mostly in combination with Snowflake.
- 20. Detectify
- 21. Elasticsearch
Distributed Search Engine, mainly a RESTful wrapper around Apache Lucene. We rely on it in most of our Logging-Infrastructure.
- 22. Snowflake
Snowflake offers a cloud-based data storage and analytics service, commonly referred to as 'data warehouse-as-a-service'. We use Snowflake in many of our data projects, and often in combination with dbt
- 23. Tideways
- 48. Apache Kafka
Messaging and streaming platform for highly scalable systems.
- 49. AWS Sagemaker
- 50. Flood.io
- 51. Istio
- 71. loft.sh
Loft enables platform teams in enterprises to provide software engineers with self-service access to Kubernetes.
- 17. Apache superset
Apache Superset is an open-source software cloud-native application for data exploration and data visualization able to handle data at petabyte scale.
- 28. Gitlab
Gitlab is one of our bread and butter tools. For years we have been using the Community Edition for source code management, CI/CD and package management, and continue to be convinced by the growing feature set.
- 29. Grafana
Composable Platform for monitoring and observability which we use in a lot of our projects to monitor our infrastructure
- 30. Harbor
A cloud native registry project that stores, signs, and scans content. Harbor extends the open source Docker Distribution by adding the functionalities usually required by users such as security, identity and management. We use it in projects which are not running in the cloud.
- 31. Helm
Helm is a package manager for Kubernetes and helps us manage our applications running in a Kubernetes cluster. Using Helm charts helps us to standardize workflows and reduces complexity.
- 32. Kubernetes
- 33. prodigy
- 34. Prometheus
Prometheus is a software application used for event monitoring and alerting. It records real-time metrics in a time series database (allowing for high dimensionality) built using a HTTP pull model, with flexible queries and real-time alerting.
- 35. RabbitMQ
- 36. Redis
- 37. Solr
- 38. Sonarqube
- 39. Snowplow
Snowplow Analytics is an open-source enterprise event-level analytics platform that enables data collection from multiple platforms for advanced data analytics. At the moment we see Snowplow as the most suitable building block for data collection, in order to be able to detach from large paid analytics products.
- 40. webpack
- 59. Clair
- 60. Cucumber
- 61. Cypress
Cypress is a fast and reliable tool for testing anything that runs in a browser. The developer friendly interface and its possibilities around time travelling while making it easy to debug makes this our tool of choice in that area.
- 62. DVC
- 63. Elastic Stack
With the Elastic Stack, we can reliably and securely capture data from any source and in any format, and then search, analyze, and visualize it in real time.
- 64. GitHub Actions
GitHub Actions helps you build, test, and deploy applications, but you can also use it to automate other tasks common to your developer workflows: triaging and managing issues, automating releases, collaborating with your user base, and more.
- 65. Kiali
Kiali is a management console for Istio-based service mesh. It provides dashboards, observability and lets you to operate your mesh with robust configuration and validation capabilities.
- 66. Rancher
Open-source multi-cluster orchestration platform, lets operations teams deploy, manage and secure enterprise Kubernetes. We use it in our data center for the setup of our internal toolings.
- 67. Skaffold
Skaffold supports us in our Kubernetes and Helm landscapes by helping us build, push and deploy applications. Currently we see it particularly well suited for programming languages that do not need to be compiled.
- 68. Renovate
We use Renovate (https://renovatebot.com/) to update our project dependencies using pull requests. This helps keeping track of outdated dependencies and minimises the work needed for updates.
- 69. Wiki.JS
- 77. Alternatives for Docker for Desktop
We evaluate tools like Podman, Minikube, or Rancher Desktop to see if they are suitable alternatives for Docker for Desktop. However Docker for Desktop might remain the tool of choice after all.
- 78. Atlantis
Atlantis is a tool which promises to help automating a pull request workflow for deploying terraform code with GitLab/Github pipelines. Thus far we have tried implementing such a workflow using only the pipelines themselves. Past efforts have proven to be difficult to mange with Terraform repositories that have multiple independently-managed infrastructure stacks. Atlantis supposedly includes functionality that detects and deploys only modified stacks and might help us build a better workflow around Terraform.
- 79. greatexpectations
- 80. Hashicorp Vault
We have long separated keys and secrets from source code management in our development process. We already use the in-house service KMS in our AWS projects for the Secrets as a Service approach. With Hashicorp Vault, we would also like to use a tool for the central management of secrets where we do not operate in the AWS Cloud.
- 81. Open Policy Agent
Open Policy Agent is a general policy engine that can uniformly implement policy across the entire stack. OPA provides a high-level declarative language that allows you to specify policies as code.
- 82. vcluster
Virtual clusters are fully working Kubernetes clusters that run on top of other Kubernetes clusters. Compared to fully separate 'real' clusters, virtual clusters do not have their own node pools. Instead, they are scheduling workloads inside the underlying cluster while having their own separate control plane.
- 85. Capistrano
- 86. Chef
- 87. Selenium
Selenium is still one of the standard tools for testing web interfaces. However, we have had very positive experiences with the cypress framework and would currently prefer it for new projects.
- 24. Clean Code
- 25. OpenAPI
- 26. Performance Testing Frontend
- 27. Pipelines as code
The Pipeline as code technique advocates that the deployment pipeline configuration for building, testing, and deploying our applications or infrastructure should be treated as code.They should be placed under source control and best modularized into reusable components.
- 52. C4Model
The C4 model is an 'abstraction-first' approach to diagramming software architecture, based upon abstractions that reflect how software architects and developers think about and build software. The small set of abstractions and diagram types makes the C4 model easy to learn and use.
- 53. Container Security Scanning
- 54. Continuous Delivery
- 55. Load Testing
- 56. OpenTelemetry
A vendor-agnostic API to help developers easily instrument tracing into their code base.
- 57. Software Composition Analyses
- 58. Static Application Security testing (SAST)
- 72. Domain Driven Design
- 73. Functional programming
Functional Programming is a paradigm which is increasingly used in modern programming languages. Pure functions and immutability allow to write easily understandable, modular and fault tolerant code. We use functional programming to limit side effects and to be able to combine our code modules easily and test them well.
- 74. Infrastructure as code Testautomation
We rely on IaC with Terraform in our cloud projects. The larger the infrastructures there become, the more important automated testing of the landscapes created with code becomes here as well. Terratest currently seems to be emerging as a tool for this.
- 75. Monorepo
In many projects, we make more intensive use of the capabilities of various package managers and the associated build and deployment workflows. At one point or another, however, we also get into the so-called 'dependency hell' with these approaches. When the opportunity arises, we would like to evaluate in a suitable project whether the approach of a monorepository can support us here. We are also thinking about the support of tools such as lerna.js.
- 76. SBOM
In order to keep track of components used in our projects we look into mechanisms to create and maintain a Software Bill of Material. This will help when checking for vulnerabilities and license conformance.