Skip to main content

28 posts tagged with "K8s edge orchestration"

View All Tags

· 6 min read
KubeEdge SIG Release

On January 23, 2024, KubeEdge released v1.16. The new version introduces several enhanced features, significantly improving cluster upgrades, cluster usability, and edge device management.

v1.16 What's New​

Release Highlights​

Support Cloud and Edge Components Upgrade​

The Cloud side and Edge side Upgrade capability is comprehensively enhanced in v1.16. Users can upgrade the cloud side components with Keadm tool, and upgrade edge nodes with the API through Kubernetes API-Server.

  • Cloud upgrade

    Keadm supports the Cloud Upgrade command, and users can easily upgrade cloud components.

    Example:

    keadm upgrade cloud --advertise-address={advertise-address} --kubeedge-version=v1.16.0

  • Edge upgrade

    In KubeEdge v1.16, the node upgrade API was implemented. Users can remotely upgrade edge nodes in batches. The cloud-edge task architecture handles upgrade task flow and supports unified timeout processing, concurrency control, and subtask management, among other capabilities.

    Upgrade API Example:

    apiVersion: operations.kubeedge.io/v1alpha1
    kind: NodeUpgradeJob
    metadata:
      name: upgrade-example
      labels:
        description: upgrade-label
    spec:
      version: "v1.16.0"
      checkItems:
        - "cpu"
        - "mem"
        - "disk"
      failureTolerate: "0.3"
      concurrency: 2
      timeoutSeconds: 180
      labelSelector:
        matchLabels:
          "node-role.kubernetes.io/edge": ""
          node-role.kubernetes.io/agent: ""

  • KubeEdge version compatibility testing

    KubeEdge v1.16 provides KubeEdge version compatibility testing, which avoids problems caused by incompatible cloud-edge versions during the upgrading process.

Refer to the link for more details. (#5330, #5229, #5289)

Alpha Implementation of Images PrePull on Edge Nodes​

In scenarios with unstable network or limited edge bandwidth, deploying or updating edge applications often results in high failure rates or reduced efficiency, especially with large-scale edge nodes.

Images PrePull feature has been introduced in v1.16. Users can perform batch images prepull on large-scale edge nodes with ImagePrePullJob API when the network is stable, to improve the success rate and efficiency of batch edge applications deploying and updating.

ImagePrePull API Example:

apiVersion: operations.kubeedge.io/v1alpha1
kind: ImagePrePullJob
metadata:
  name: imageprepull-example
  labels:
    description:ImagePrePullLabel
spec:
  imagePrePullTemplate：
    images:
      - image1
      - image2
    nodes：
      - edgenode1
      - edgenode2
    checkItems:
      - "disk"
    failureTolerate: "0.3"
    concurrency: 2
    timeoutSeconds: 180
    retryTimes: 1

Refer to the link for more details. (#5310, #5331)

Support Installing Windows-based Edge Nodes with Keadm​

KubeEdge has supported the edge node running on Windows Server 2019 in v1.15, extending KubeEdge to the Windows ecosystem and expanding its use cases and ecosystem.

In this release, Windows-based Edge Nodes can be installed and registered to cloud with the installation tool Keadm, providing convenience for the application of KubeEdge in Windows OS.

Refer to the link for more details. (#4968)

Add Compatibility Tests for Multiple Runtimes​

The e2e test of KubeEdge v1.16 has integrated compatibility tests for multiple container runtimes. Currently, four container runtime compatibility tests have been added, including containerd, docker, cri-o, and isulad.

Refer to the link for more details.(#5321)

Support More Deployment Fields to the EdgeApplication Overrides​

In previous versions, only replicas and image of the EdgeApplication could be overridden. In this release, we support overriding more Deployment fields: env, command, args and resources.

Refer to the link for more details.(#5038)

Support Mapper Upgrade​

Build mapper upgrade framework. Users can upgrade the mapper by changing the referenced mapper-framework package version.

  • Mapper-framework code decouple

    The code in mapper-framework was decoupled into user-layer code and business-layer code, and create the kubeedge/mapper-framework repo to store the business layer code.

  • Mapper upgrade framework

    Update the way mapper-framework generates mapper projects. The current execution script will only generate user-level code through dependent references. When the mapper project needs to be upgraded, it can be directly made by changing the version of mapper-framework package.

Refer to the link for more details.(#5308, #5326)

Integrate Redis and TDengine Database in DMI Data Plane​

Integrate redis and tdengine database in DMI data plane. The mapper project generated by mapper-framework has build-in ability to push data to redis and tdengine database. Users can push data directly through configuring device instance files.

Database Field Definition:

type DBMethodRedis struct {
    // RedisClientConfig of redis database
    // +optional
    RedisClientConfig *RedisClientConfig `json:"redisClientConfig,omitempty"`
}
type RedisClientConfig struct {
    // Addr of Redis database
    // +optional
    Addr string `json:"addr,omitempty"`
    // Db of Redis database
    // +optional
    DB int `json:"db,omitempty"`
    // Poolsize of Redis database
    // +optional
    Poolsize int `json:"poo lsize,omitempty"`
    // MinIdleConns of Redis database
    // +optional
    MinIdleConns int `json:"minIdleConns,omitempty"`
}
type DBMethodTDEngine struct {
    // tdengineClientConfig of tdengine database
    // +optional
    TDEngineClientConfig *TDEngineClientConfig `json:"TDEngineClientConfig,omitempty"`
}
type TDEngineClientConfig struct {
    // addr of tdEngine database
    // +optional
    Addr string `json:"addr,omitempty"`
    // dbname of tdEngine database
    // +optional
    DBName string `json:"dbName,omitempty"`
}

Refer to the link for more details.(#5064)

New USB Camera Mapper​

Based on the mapper and dmi framework in KubeEdge v1.15.0, a mapper for USB cameras has been developed, which supports data push to Influxdb, mqtt, and http. It has been successfully applied in practice.

Refer to the link for more details.(#122)

Keadm’s Enhancement​

  • When using Keadm join in kubeEdge v1.16, it supports the selection of communication protocols for edge nodes and cloud center nodes. The cloud edge communication protocol is configured through the parameter --hub-protocol, and currently supports two communication protocols: websocket and quic.

    note

    When the --hub-protocol parameter is configured as quic, it is necessary to set the port of the parameter --cloudcore-ipport to 10001 and modify configmap in cloudcore to open the quic protocol.

    Refer to the link for more details.(#5156)

  • In KubeEdge v1.16, it is already supported for Keadm to complete edgecore deployment through Keadm join without installing the CNI plugin, decoupling the deployment of edge nodes from the CNI plugin. At the same time, this feature has been synchronized to v1.12 and later versions.

    note

    If the application deployed on edge nodes needs to use container networks, it is still necessary to install the CNI plugin after deploying edgecore.

    Refer to the link for more details.(#5196)

Upgrade Kubernetes Dependency to v1.27.7​

Upgrade the vendered kubernetes version to v1.27.7, users are now able to use the feature of new version on the cloud and on the edge side.

Refer to the link for more details. (#5121)

Important Steps before Upgrading​

  • Now we use DaemonSet to manage the mqtt broker mosquitto. You need to consider whether to use the static pod managed mqtt broker in the edge node or use the DaemonSet managed mqtt broker in the cloud, they cannot coexist and there will be port conflicts. You can read the guide For edge node low version compatibility in #5233.

  • In this release, the flag with-mqtt will be set to deprecated and default to false, but will not be removed. After v1.18, the code related to static pod management will be removed in the edge, and the flag with-mqtt no longer supported.

· 4 min read
KubeEdge SIG Release

On Oct 13, 2023, KubeEdge released v1.15. The new version introduces several enhanced features, significantly improving support for Windows-based edge nodes, device management, and data plane capabilities.

v1.15 What's New​

Release Highlights​

Support Windows-based Edge Nodes​

Edge computing involves various types of devices, including sensors, cameras, and industrial control devices, some of which may run on the Windows OS. In order to support these devices and use cases, supporting Windows Server nodes is necessary for KubeEdge.

In this release, KubeEdge supports the edge node running on Windows Server 2019, and supports Windows container running on edge node, thereby extending KubeEdge to the Windows ecosystem and expanding its use cases and ecosystem.

Refer to the link for more details. (#4914, #4967)

New v1beta1 version of Device API​

The device API is updated from v1alpha2 to v1beta1, in v1beta1 API updates include:

  • The built-in protocols incude Modbus, Opc-UA and Bluetooth are removed in device instance, and the built-in mappers for these proytocols still exists and will be maintained and updated to latest verison.

  • Users must define the protocol config through CustomizedValue in ProtocolConfig.

  • DMI date plane related fields are added, users can config the collection and reporting frequency of device data, and the destination to whcih(such as database, httpserver) data is pushed.

  • Controls whether to report device data to cloud.

Refer to the link for more details. (#4983)

Support Alpha version of DMI DatePlane and Mapper-Framework​

Alpha version of DMI date plane is supported, DMI date plane is mainly implemented in mapper, providing interface for pushing data, pulling data, and storing data in database.

To make writing mapper easier, a mapper development framework subproject Mapper-Framework is provided in this release. Mapper-Framework provides mapper runtime libs and tools for scaffolding and code generation to bootstrap a new mapper project. Users only need to run a command make generate to generate a mapper project, then add protocol related code to mapper.

Refer to the link for more details. (#5023)

Support Kubernetes native Static Pod on Edge Nodes​

Kubernetes native Static Pod is supported on edge node in this release. Users can create pods on edge nodes by place pod manifests in /etc/kubeedge/manifests, same as that on the Kubernetes node.

Refer to the link for more details. (#4825)

Support more Kubernetes Native Plugin Running on Edge Node​

Kubernetes non-resource kind request /version is supported from edge node, users now can do /version requests in edge node from metaserver. In addition, it can easily support other non-resource kind of requests like /healthz in edge node with the curent framework. Many kubernetes plugins like cilium/calico which depend on these non-resource kind of requests, now can run on edge nodes.

Refer to the link for more details. (#4904)

Upgrade Kubernetes Dependency to v1.26.7​

Upgrade the vendered kubernetes version to v1.26.7, users are now able to use the feature of new version on the cloud and on the edge side.

Refer to the link for more details. (#4929)

Important Steps before Upgrading​

  • In KubeEdge v1.15, new v1beta1 version of device API is incompatible with earlier versions of v1alpha1, users need to update the device API yamls to v1bata1 if you want to use v1.15.

  • In KubeEdge v1.15, users need to upgrade the containerd to v1.6.0 or later. Containerd minor version 1.5 and older will not be supported in KubeEdge v1.15.
    Ref: https://kubernetes.io/blog/2022/11/18/upcoming-changes-in-kubernetes-1-26/#cri-api-removal

  • In KubeEdge v1.14, EdgeCore has removed the dockeshim support, so users can only use remote type runtime, and uses containerd runtime by default. If you want to use docker runtime in v1.15, you also need to first set edged.containerRuntime=remote and corresponding docker configuration like RemoteRuntimeEndpoint and RemoteImageEndpoint in EdgeCore, then install the cri-dockerd tools as docs below: https://github.com/kubeedge/kubeedge/issues/4843

· 3 min read
KubeEdge SIG Release

On July 1, 2023, KubeEdge released v1.14. The new version introduces several enhanced features, significantly improving security, reliability, and user experience.

v1.14 What's New​

Release Highlights​

Support Authentication and Authorization for Kube-API Endpoint for Applications On Edge Nodes​

The Kube-API endpoint for edge applications is implemented through MetaServer in edegcore. However, in previous versions, the authentication and authorization of Kube-API endpoint are performed in the cloud, which prevents authentication and authorization especially in offline scenarios on the edge node.

In this release, the authentication and authorization functionalities are implemented within the MetaServer at edge, which allows for limiting the access permissions of edge applications when accessing Kube-API endpoint at edge.

Refer to the link for more details. (#4802)

Support Cluster Scope Resource Reliable Delivery to Edge Node​

The cluster scope resource can guarantee deliver to the edge side reliably since this release, especially include using list-watch global resources, the cluster scope resource can be delivered to the edge side reliably, and the edge applications can work normally.

Refer to the link for more details. (#4758)

Upgrade Kubernetes Dependency to v1.24.14​

Upgrade the vendered kubernetes version to v1.24.14, users are now able to use the feature of new version on the cloud and on the edge side.

note

The dockershim has been removed, which means users can't use docker runtime directly in this release.

Refer to the link for more details. (#4789)

Support Kubectl Attach to Container Running on Edge Node​

KubeEdge already support kubectl logs/exe command, kubectl attach is supported in this release. kubectl attach command can attach to a running container at edge node. Users can execute these commands in the cloud and no need to operate on the edge nodes.

Refer to the link for more details. (#4734)

Alpha version of KubeEdge Dashboard​

KubeEdge dashboard provides a graphical user interface (GUI) for managing and monitoring your KubeEdge clusters. It allows users to manage edge applications running in the cluster and troubleshoot them.

Refer to the link for more details. (https://github.com/kubeedge/dashboard)

Important Steps before Upgrading​

  • On KubeEdge v1.14, EdgeCore has removed the dockeshim support, so users can only use remote type runtime, and uses containerd runtime by default. If you want to use docker runtime, you must first set edged.containerRuntime=remote and corresponding docker configuration like RemoteRuntimeEndpoint and RemoteImageEndpoint in EdgeCore, then install the cri-dockerd tools as docs below: https://github.com/kubeedge/kubeedge/issues/4843

· 3 min read
KubeEdge SIG Release

On Jan 18, 2023, KubeEdge released v1.13. The new version introduces several enhanced features, significantly improving performance, security, and edge device management.

v1.13 What's New​

Performance Improvement​

  • CloudCore memory usage is reduced by 40%, through unified generic Informer and reduce unnecessary cache. (#4375, #4377)

  • List-watch dynamicController processing optimization, each watcher has a separate channel and goroutine processing to improve processing efficiency (#4506)

  • Added list-watch synchronization mechanism between cloud and edge and add dynamicController watch gc mechanism (#4484)

  • Removed 10s hard delay when offline nodes turn online (#4490)

  • Added prometheus monitor server and a metric connected_nodes to cloudHub. This metric tallies the number of connected nodes each cloudhub instance (#3646)

  • Added pprof for visualization and analysis of profiling data (#3646)

  • CloudCore configuration is now automatically adjusted according to nodeLimit to adapt to the number of nodes of different scales (#4376)

Security Improvement​

  • KubeEdge is proud to announce that we are digitally signing all release artifacts (including binary artifacts and container images). Signing artifacts provides end users a chance to verify the integrity of the downloaded resource. It allows to mitigate man-in-the-middle attacks directly on the client side and therefore ensures the trustfulness of the remote serving the artifacts. By doing this, we reached the SLSA security assessment level L3 (#4285)

  • Remove the token field in the edge node configuration file edgecore.yaml to eliminate the risk of edge information leakage (#4488)

Upgrade Kubernetes Dependency to v1.23.15​

Upgrade the vendered kubernetes version to v1.23.15, users are now able to use the feature of new version on the cloud and on the edge side.

Refer to the link for more details. (#4509)

Modbus Mapper based on DMI​

Modbus Device Mapper based on DMI is provided, which is used to access Modbus protocol devices and uses DMI to synchronize the management plane messages of devices with edgecore.

Refer to the link for more details. (mappers-go#79)

Support Rolling Upgrade for Edge Nodes from Cloud​

Users now able to trigger rolling upgrade for edge nodes from cloud, and specify number of concurrent upgrade nodes with nodeupgradejob.spec.concurrency. The default Concurrency value is 1, which means upgrade edge nodes one by one.

Refer to the link for more details. (#4476)

Test Runner for conformance test​

KubeEdge has provided the runner of the conformance test, which contains the scripts and related files of the conformance test.

Refer to the link for more details. (#4411)

EdgeMesh: Added configurable field TunnelLimitConfig to edge-tunnel module​

The tunnel stream of the edge-tunnel module is used to manage the data stream state of the tunnel. Users can obtain a stable and configurable tunnel stream to ensure the reliability of user application traffic forwarding.

Users can configure the cache size of tunnel stream according to TunnelLimitConfig to support larger application relay traffic.

Refer to the link for more details. (#399)

Cancel the restrictions on the relay to ensure the stability of the user's streaming application or long link application.

Refer to the link for more details. (#400)

Important Steps before Upgrading​

  • EdgeCore now uses containerd runtime by default on KubeEdge v1.13. If you want to use docker runtime, you must set edged.containerRuntime=docker and corresponding docker configuration like DockerEndpoint, RemoteRuntimeEndpoint and RemoteImageEndpoint in EdgeCore.