Valiton Technology Radar
Stand: März 2023
In unserem Technology Radar haben wir für uns relevante Technologien und Prozesse zusammengefasst und für uns auf Basis unserer täglichen Arbeit bewertet. Dinge, die wir in neuen Projekten möglichst nicht mehr einsetzen möchten (HOLD), sind hier genauso aufgeführt wie Elemente von deren Relevanz wir auch für die Zukunft überzeugt sind (ADOPT).
Languages & Frameworks
Adopt
- 1. Apache Spark
Apache Spark is an open-source distributed computing framework designed for fast and efficient processing and analysis of large datasets. It supports multiple programming languages and provides excellent flexibility for integration. Leveraging Spark helps organizations to quickly process data from sources such as databases, streaming services, and cloud storage, and generate insights more quickly through distributed computing.
- 2. Cube.js
Cube.js is a platform for building analytical web applications that leverage an organization’s existing database. It works with structured data sources, enabling powerful tools for transforming and pre-aggregating data. The platform is highly flexible, allowing organizations to easily integrate it into existing applications and platforms. We recommend leveraging Cube.js, as it provides a simple yet powerful way to build scalable analytical applications that can generate insights quickly and efficiently.
- 3. Doctrine
Our choice for object-relational mapping (ORM) library for PHP is Doctrine. It provides a powerful and flexible solution to map object-oriented code to a relational database. Its high degree of flexibility allows for easy integration with existing applications and platforms. With Doctrine, managing data and integrating with databases is reliable and efficient.
- 4. Eslint
For static code analysis, identifying and fixing errors, and maintaining consistent coding standards, we recommend Eslint. It is a powerful tool that ensures high-quality code, making it easy to detect issues before they become problems. Eslint is highly customizable and can be used with any JavaScript project, making it an ideal choice for developers looking for a reliable and efficient way to maintain code quality.
- 5. FastAPI
FastAPI is a high-performance Python web framework for building APIs quickly and efficiently. It offers a set of tools for creating scalable APIs that are easy to learn and use. FastAPI's out-of-the-box support for asynchronous programming makes it a great choice for building real-time applications. With FastAPI, building APIs that can handle high traffic loads is simple and efficient.
- 6. Gatsby.js
For building fast and efficient websites that are tailored to specific needs, we recommend Gatsby.js. It provides a modern web framework that is highly modular and customizable, making it a great choice for developers looking for flexibility. Based on React and GraphQL, Gatsby.js is easy to integrate with existing applications and platforms. With Gatsby.js, building modern, scalable websites is reliable and efficient.
- 7. gensim
For topic modeling and natural language processing (NLP), we recommend gensim. It is a Python library that provides a reliable and efficient way to extract insights and patterns from large text datasets. Gensim is easy to integrate with existing applications and platforms, making it a great choice for organizations looking to build custom NLP applications.
- 8. Huggingface Transformers
Huggingface Transformers is an open-source library for natural language processing (NLP). It offers a wide range of pre-trained models and tools for analyzing and processing text data, making it easy to generate insights and predictions. Huggingface Transformers is easy to integrate with existing applications and platforms, providing a simple and efficient way to build NLP applications.
- 9. Jest
For testing React applications, we recommend Jest. It is a JavaScript testing framework that provides a reliable and efficient way to test code, ensuring high-quality code and proper functioning of React applications. Jest is easy to learn and use, making it an ideal choice for developers who are new to testing frameworks.
- 10. Next.js
Next.js is flexible JavaScript framework for creating scalable applications. It enables server-side rendering and provides an intuitive page-based routing system, making it easy to create complex, rich web experiences. Next.js is modular, allowing organizations to quickly and easily integrate custom modules, including APIs, libraries, and plugins. Its powerful development tools support popular databases, allowing for faster iteration and deployment. With Next.js, building dynamic, feature-rich applications is simple and efficient.
- 11. Pandas
Pandas is a Python library used for data manipulation and analysis. It provides data structures for efficiently storing and manipulating large data sets, as well as tools for data cleaning, merging, and filtering. Pandas is widely used in data science and machine learning projects due to its flexibility and ease of use. We recommend using Pandas, as it simplifies and accelerates data analysis tasks.
- 12. PyTorch
PyTorch is an open-source machine learning library for efficient computation and development of deep learning models. It is used for a wide range of tasks, from training models to producing complex neural networks. PyTorch’s flexible architecture makes it easy to integrate with existing applications and platforms. It also has excellent support for GPUs, enabling organizations to take advantage of hardware acceleration for model training. We recommend leveraging PyTorch, as it provides a powerful, reliable, and efficient way to develop and train deep learning models.
- 13. React
React is a popular JavaScript library for building user interfaces. It provides a component-based architecture, making it easy to create reusable UI elements that can be used across projects. React also supports server-side rendering and has excellent performance and scalability, making it ideal for large-scale projects. We suggest using React, as it simplifies UI development and provides a flexible and scalable foundation for web applications.
- 14. React Testing Library
React Testing Library is a lightweight library used for testing React components. It provides a simple and intuitive API for testing user interactions and state changes, making it easy to write maintainable and robust tests. React Testing Library also promotes good testing practices, such as testing components from the user's perspective, resulting in more reliable and maintainable tests. We suggest using React Testing Library, as it simplifies and improves the quality of React component testing.
- 15. Sass
Sass is a CSS preprocessor that allows developers to write more efficient and maintainable CSS code. It provides a range of features, such as variables, mixins, and nesting, that make it easier to write and organize CSS. Sass also includes powerful features for generating CSS, such as functions and control structures, enabling developers to create more flexible and reusable styles. We recommend using Sass, as it simplifies CSS development and improves code organization and maintainability.
- 16. Serverless Framework
We use the serverless framework pretty much everywhere we deploy AWS Lambda, for example. We also try to stay as provider independent as possible in the code.
- 17. spacy
Spacy is a free, open-source library for advanced natural language processing in Python. It's fast and efficient, with high accuracy rates. We use it extensively in our data projects, such as text classification and entity recognition.
- 18. Symfony
Symfony is a popular PHP web application framework used for building complex, high-performance web applications. It's been around for over a decade and has a strong community of developers supporting it. We've used it in several projects and find it reliable and easy to work with.
- 19. Tailwind CSS
Tailwind CSS follows the utility-first approach of Atomic Design Methodology. Unlike heavyweight frameworks like Bootstrap and Bulma, Tailwind CSS classes with built-in design system can be flexibly combined making it easy to create custom designs. Tailwind works with any JavaScript framework or even plain HTML, often reduces the amount of CSS required by a fraction and produces very small stylesheets through CSS purging (often < 10KB).
- 20. Terraform
Terraform is an Infrastructure as code (IaC) tool that lets you create, modify, and version infrastructures securely and efficiently. We use it extensively in all our cloud projects, and actually deploy any type of infrastructure in the cloud only using terraform.
- 21. Vue.js
Vue.js is a progressive JavaScript framework for building user interfaces. It's lightweight and easy to integrate into existing projects. We've used it in several projects and found it to be a great choice for building dynamic, responsive user interfaces.
Trial
- 53. LangChain
LangChain is an advanced framework that empowers the development of applications powered by Large Language Models (LLMs). It seamlessly supports renowned LLMs such as ChatGPT, Vertex, Cohere, aleph alpha, and many others. With LangChain, developers can leverage a variety of reusable components, including prompt templates, Chat Histories Memory, and tools for incorporating results from vector searches and external sources like Google, Wikipedia, and Wolfram into prompts. These versatile components and tools can be effortlessly chained together to orchestrate the LLM pipeline, all through intuitive and standardized interfaces.
Assess
- 68. Headless CMS
Headless CMS systems provide an alternative to traditional CMS platforms, allowing organizations to manage, store, and deliver digital content more flexibly and efficiently. They decouple the backend content repository from the frontend user experience, enabling custom publishing and integrations with software languages and content partners. We would like to evaluate multiple headless CMS tools such as Strapi to identify the right platform for our organization.
- 69. Polars
Polars is a DataFrame library for Python and also Rust. It is one of the fastest Panda alternatives by parallelize processing and using all available cpu cores. It shines with impressive benchmarks in performance and memory usage. It has a mature feature set, but lagging compared to panda, eg plotting graphs. Therefore, it very good alternative for automating data pipelines where processing speed is of the essence.
- 70. Wasm & WASI
WebAssembly (Wasm) is a binary format to compile portable client and server web applications. Wasm is an open standard of the W3C. It was created to run in the browser and is supported by all major browsers. However it becomes more and more populate on the on the server side. It associated with the 3rd wave of cloud computing (after VM and containers) and as a replacement for typical containers because of its fast startup time, lightness (compared to containers), cross platform capability and secure sandboxing. Also Docker has released WASM support (beta). To run on the server the source code must be compiled for WASI (WebAssembly System Interface) and can than be executed in a runtime environment for WebAssembly like WasmEdge (supported by Docker), Wasmer or Wasmtime (Bytecode Alliance of major companies). WASI supporting the following programming languages: Rust, Go, C/C++, Python, .Net and JS.
Hold
- 79. Angular
Framework for building modern mobile and desktop applications. Due to the larger community, we decided to focus on React and Vue.js, and therefore put Angular on hold.
Platforms
Adopt
- 22. Apache Airflow
Apache Airflow is an open-source platform used to programmatically author, schedule, and monitor workflows. We use it heavily in most data projects, such as ETL pipelines, as it offers great flexibility and extensibility.
- 23. Apache Kafka
Apache Kafka is an open-source distributed event streaming platform used for building fault-tolerant architectures for real-time streaming data. It can be used to collect, store, distribute, and analyze massive amounts of data from numerous sources. Kafka is highly configurable, offering robust APIs for both producers and consumers, as well as security and scalability support. Leveraging Kafka can help us to process and manage streaming data more effectively and efficiently.
- 24. AWS Organizations
We use AWS Organizations to maintain and govern our AWS infrastructure and the one of our customers. It enables us to simplify plenty of everyday tasks and supports us in applying our security standards.
- 25. AWS Sagemaker
AWS Sagemaker is a cloud-based machine learning platform that provides a fully managed environment for training, deploying, and managing models. It enables our data scientists to quickly build, train, and deploy high-quality machine learning models without needing to manage the underlying infrastructure. SageMaker includes extensive APIs, algorithms, architectures, and tools to support every step of the machine learning process.
- 26. dbt
dbt is a command-line tool that enables data analysts and engineers to transform data in their warehouse more effectively. We use it mostly in combination with Snowflake to build high-quality data pipelines and models.
- 27. Detectify
We use Detectify, a web application security scanner, regularly to ensure the security of our web applications. It helps us identify vulnerabilities and is a reliable and effective tool.
- 28. Elasticsearch
Elasticsearch is a distributed search engine that we use mainly as a RESTful wrapper around Apache Lucene. We rely on it in most of our logging infrastructure and find it to be a reliable and effective technology.
- 29. Snowflake
Snowflake is a cloud-based data storage and analytics service that we use in many of our data projects. It's often used in combination with dbt, and we have found it to be a reliable and effective technology.
- 30. Tideways
Tideways is a PHP performance monitoring and profiling solution that we use to identify performance bottlenecks in our PHP applications.
- 31. Weaviate
Weaviate is an open-source vector database. It allows you to store data objects and vector embeddings from your favorite ML-models, and scale seamlessly into billions of data objects. A vector data base builds the backbone of a vector search or a generative search.
Trial
- 54. Istio
Istio is a mature platform that offers reliable scalability, security and performance in managing microservices across a network. Organizations should carefully evaluate its various features and compare them to potential alternatives before determining whether it is the right choice for the task at hand. In addition, we also recommend considering if using Istio will be cost effective and whether it can provide any additional benefits to the service design.
Assess
- 71. API Gateways
API Gateways provide centralized control, visibility, and enhanced security for an organization’s APIs. They enable a business to control who has access to its APIs, which operations a consumer can perform, and metrics like rate-limiting and throttling to prevent resources from being overloaded. Other features include traffic control, authorization, authentication, request validation, and analytics to help optimize performance. We would like to evaluate various API management and gateway solutions in order to identify the platform best suited to meet our needs
Hold
- 80. loft.sh
Loft enables platform teams in enterprises to provide software engineers with self-service access to Kubernetes. We currently see no usecase for us and our clients, therefore we put the item to hold for now.
Tools
Adopt
- 32. Apache superset
Apache Superset is an open-source software cloud-native application for data exploration and data visualization. It is capable of handling data at petabyte scale, through our experience, we have established that this technology is dependable and efficient
- 33. Gitlab
Gitlab is one of our primary tools for source code management, CI/CD and package management. We have been using the Community Edition for years and continue to be impressed by its growing feature set.
- 34. Grafana
Grafana is a composable platform for monitoring and observability which we use in many of our projects to monitor our infrastructure. It has consistently demonstrated its reliability and effectiveness in our experience, making it a go-to tool for us.
- 35. Harbor
Harbor is a cloud native registry project that enhances the open source Docker Distribution by adding security, identity, and management functionalities. It enables us to store, sign, and scan content for our projects that are not running in the cloud.
- 36. Helm
Helm is a package manager for Kubernetes and helps us manage our applications running in a Kubernetes cluster. Using Helm charts helps us to standardize workflows and reduces complexity.
- 37. Kubernetes
Kubernetes is an open-source container orchestration platform that automates the deployment, scaling, and management of containerized applications. We have found it to be a reliable and effective technology for managing our applications.
- 38. prodigy
Prodigy is an annotation tool for creating training data for machine learning models. We use it to annotate data for our various machine learning projects.
- 39. Prometheus
Prometheus is a software application used for event monitoring and alerting. It records real-time metrics in a time series database (allowing for high dimensionality) built using a HTTP pull model, with flexible queries and real-time alerting.
- 40. RabbitMQ
RabbitMQ is a message broker that enables applications to send and receive messages using a variety of messaging protocols. It supports multiple messaging patterns, including point-to-point, publish/subscribe, and request/reply. RabbitMQ is highly available and can scale to handle large amounts of messaging traffic, making it a reliable and efficient choice for message-based architectures.
- 41. Redis
Redis is an in-memory key-value store that can be used as a database, cache, and message broker. It supports a wide range of data structures, including strings, hashes, lists, sets, and sorted sets. Redis is highly available and provides automatic failover, making it a reliable choice for applications that require high availability. Its rich set of features and APIs make it a versatile tool for building complex applications.
- 42. Renovate
We use Renovate (https://renovatebot.com/) to update our project dependencies using pull requests. It automatically applies all applicable patches and security updates, while filtering out risky changes. This helps keeping track of outdated dependencies and minimises the work needed for updates.
- 43. Snowplow
Snowplow Analytics is an open-source enterprise event-level analytics platform that enables data collection from multiple platforms for advanced data analytics. At the moment we see Snowplow as the most suitable building block for data collection, in order to be able to detach from large paid analytics products.
- 44. Solr
Solr is a widely used, open source search platform that allows for powerful full-text search, hit highlighting, faceted search, dynamic clustering, database integration, and more. We use Solr for search and retrieval in our projects, as it provides efficient and accurate results.
- 45. Sonarqube
SonarQube is an open-source platform for continuous inspection of code quality. It provides metrics, code coverage, and code duplication analysis, as well as automated code reviews, which help us maintain code quality and reduce technical debt.
- 46. Wiki.JS
An open-source system to help create and manage wikis quickly and easily. It provides an intuitive user interface, version control, and extensible plugin architecture. It is highly configurable with customizable templates and language support, helping organizations ensure accuracy and consistency in the documentation process. Leveraging Wiki.js helps us streamline the process of creating and managing wikis.
Trial
- 55. Alternatives for Docker for Desktop
We evaluate tools like Podman, Minikube, or Rancher Desktop to see if they are suitable alternatives for Docker for Desktop. However Docker for Desktop might remain the tool of choice after all.
- 56. Atlantis
Atlantis is a tool which promises to help automating a pull request workflow for deploying terraform code with GitLab/Github pipelines. Thus far we have tried implementing such a workflow using only the pipelines themselves. Past efforts have proven to be difficult to mange with Terraform repositories that have multiple independently-managed infrastructure stacks. Atlantis supposedly includes functionality that detects and deploys only modified stacks and might help us build a better workflow around Terraform.
- 57. Clair
We are currently evaluating Clair, an open source, static security analyzer for containers that can detect vulnerabilities in images. It uses CVE databases, provides an alarm system, and tracks infrastructure health. Clair is enterprise-grade and easy to integrate, but we are assessing its suitability for our needs by comparing it with other solutions like Trivy.
- 58. Cucumber
Cucumber BDD, an open source platform designed to assist with Behavior Driven Development (BDD). Its ordinary language parser, Gherkin, allows software behaviors to be specified with a logical language that customers can understand and quickly execute feature documentation in a business-facing language. We are assessing its overall suitability for our needs and evaluating whether it fits into our and our clients development workflows.
- 59. Cypress
Cypress is a fast and reliable tool for testing anything that runs in a browser. The developer friendly interface and its possibilities around time travelling while making it easy to debug makes this currently our tool of choice.
- 60. DVC
Data Version Control is a system designed specifically for machine learning projects, enabling developers to securely track, protect, share and reproduce data sets, functions, pipelines and more throughout the project lifecycle. DVC reduces onboarding time and costs, making it worth a trial for its ability to streamline data version control.
- 61. Elastic Stack
With the Elastic Stack, we can reliably and securely capture data from any source and in any format, and then search, analyze, and visualize it in real time.
- 62. Hashicorp Vault
We have long separated keys and secrets from source code management in our development process. We already use the in-house service KMS in our AWS projects for the Secrets as a Service approach. With Hashicorp Vault, we would also like to use a tool for the central management of secrets where we do not operate in the AWS Cloud. After the recent events it might also be an alternative to LastPass to address issues like sharing credentials within a team.
- 63. KubeClarity
KubeClarity is a tool for detection and management of Software Bill Of Materials (SBOM) and vulnerabilities of container images and filesystems. It scans both runtime K8s clusters and CI/CD pipelines for enhanced software supply chain security.
- 64. Vite
Vite is a new generation build tool for scaffolding and building modern javascript projects, with a rapidly growing popularity. It is of framework-agnostic. Vite takes advantage of the native ES modules support and esbuild for pre-bundling dependencies (on development and rollup on production). As a result, it can drastically reduce the build time compared to webpack, rollup and parcel. It also offers out-of-the-box support for TypeScript. Vite has replaced webpack in the vue-cli. No wonder, since Evan You created both Vite and Vue.js.
Assess
- 72. GreatExpectations
GreatExpectations is an open-source data quality framework to enable best practices and guardrails via automated data validation and profiling. We want to assess its suitability for our organization for efficient, cost-effective data quality.
Hold
- 81. Rancher
Rancher is a cloud-native platform, and it is still in a maturation process. As the platform matures, businesses might expect more reliable, secure, and cost-effective ways to deploy and scale cloud-native applications; however, until more features, tools, and resources are added, we are evaluating other possibilities for bootstrapping kubernetes clusters.
- 82. vcluster
Virtual clusters are fully working Kubernetes clusters that run on top of other Kubernetes clusters. Compared to fully separate 'real' clusters, virtual clusters do not have their own node pools. Instead, they are scheduling workloads inside the underlying cluster while having their own separate control plane. We currently see no usecase for us and our clients, therefore we put the item to hold for now.
- 83. webpack
We have placed Webpack, a JavaScript module bundler, on hold. Webpack can be complex and resource-intensive, needing a good understanding of the project’s underlying technology and structure. Poor configuration or plugin usage can lead to bloated bundles and browser incompatibility issues. Currently, we are considering alternatives like Vite to understand if they are better suited for our projects.
Techniques
Adopt
- 47. Clean Code
Clean Code is a set of principles and practices for writing code that is easy to understand, maintain, and extend. It includes naming conventions, code formatting, and the use of comments and documentation. We follow the principles of Clean Code to ensure that our code is readable, maintainable, and of high quality.
- 48. Functional programming
Functional Programming is a programming paradigm that emphasizes the use of pure functions, immutable data, and declarative style. We use functional programming to write code that is modular, testable, and easier to reason about. It also helps us to write code that is more resilient to change and easier to maintain.
- 49. OpenAPI
OpenAPI (formerly known as Swagger) is an open standard for describing APIs. It provides a way to describe the structure and functionality of APIs, which makes it easier to develop, test, and maintain them. We use OpenAPI to document our APIs, which makes it easier for developers to understand how to use them and to build applications that consume them.
- 50. Performance Testing Frontend
Performance testing of frontend applications is a technique that helps us to measure the speed and stability of our applications. It involves simulating real-world scenarios and measuring the performance of our applications under different loads. We use performance testing to ensure that our applications are fast, reliable, and can handle the traffic they receive.
- 51. Pipelines as code
The Pipeline as code technique advocates that the deployment pipeline configuration for building, testing, and deploying our applications or infrastructure should be treated as code.They should be placed under source control and best modularized into reusable components.
- 52. SBOM
In order to keep track of components used in our projects we look into mechanisms to create and maintain a Software Bill of Material. This will help when checking for vulnerabilities and license conformance.
Trial
- 65. C4Model
The C4 model is an 'abstraction-first' approach to diagramming software architecture, based upon abstractions that reflect how software architects and developers think about and build software. The small set of abstractions and diagram types makes the C4 model easy to learn and use.
- 66. Continuous Delivery
a DevOps practice of continuously delivering small, frequent updates to software applications. Through automated processes and reliable pipelines, CD enables teams to quickly and safely deliver new versions to end-users, allowing teams to take advantage of market opportunities, adjust to changes in customer requirements and reduce risk. We are currently evaluating its suitability, determining the cost and productivity improvements that CD may offer our organization.
- 67. Monorepo
In many projects, we make more intensive use of the capabilities of various package managers and the associated build and deployment workflows. At one point or another, however, we also get into the so-called 'dependency hell' with these approaches. When the opportunity arises, we would like to evaluate in a suitable project whether the approach of a monorepository can support us here. We are also thinking about the support of tools such as lerna.js.
Assess
- 73. BFF
Backend for Frontend (BFF) is an architecture that uses another backend service to filter, format, and aggregate data from multiple other microservices for the frontend. This also makes it easy to cache the data.
- 74. Data mesh
Data mesh is a concept to build decentralized data architectures. We expect that this concept helps us to limit the bottleneck of central data teams.
- 75. Generative AI for Code
There are now quite a few options for AI based support in the development process. From the commercial variant Github Copilot to the open source option GPT Code Clippy, there are several approaches. We would like to explore these and understand if and how they can improve our development processes.
- 76. Infrastructure as code Testautomation
We rely on IaC with Terraform in our cloud projects. The larger the infrastructures there become, the more important automated testing of the landscapes created with code becomes here as well. Terratest currently seems to be emerging as a tool for this.
- 77. Mutation Testing
Mutation Testing is a fault-based testing technique which provides a testing criterion called the Mutation Score Indicator (MSI). The MSI can be used to measure the effectiveness of a test set in terms of its ability to detect faults.
- 78. Policy as a Code
(PaC) is a DevOps practice that enables organizations to express their security and compliance policies as easy-to-maintain code. This approach makes policies easier to manage and update, leading to more secure and reliable operations. We are therefore assessing the suitability of PaC to benefit from its ability to streamline the enforcement of policies and reduce the overhead of manual governance.