Development¶

This chapter shows which options are available to customize the charge controller to your own needs. It starts with creating your own EVerest modules that can be cross-compiled and run on the target, up to creating fully customized firmware images that can be installed on the target. This chapter also explains how to debug issues within the EVerest environment on the target.

Preparation Checklist for Development¶

The following bullet points should be checked before starting with the development of a custom EVerest module:

Please make sure that your charge controller is started and running (see Getting Started chapter). It is recommended to connect the charge controller to the local computer via Ethernet. This makes it easier to share files (e.g. configuration, image, binary, or logging files) between the local computer and the charge controller.
The EVerest development environment should be set up on the local Linux machine. Please follow the instructions in the EVerest documentation, in particular the chapter Get started in Software. If you have problems setting up EVerest, please check the Zulip chat to see whether this problem is already known and whether solutions have already been found. To verify that everything is installed correctly, EVerest should be compiled once natively on your system.
Keep in mind that if you want to cross-compile an EVerest module, it must be compatible with the EVerest release used in the firmware running on the charge controller. Please have a look at the official EVerest documentation to learn how to check out a dedicated EVerest release. The EVerest release used by the firmware can be found in the release notes of the firmware images.

Note

All charge controllers shipped by chargebyte and the provided firmware images by chargebyte are currently based on Yocto kirkstone, which includes Python 3.10. Since there are some tooling dependencies during cross-compiling for the target platform, please use a Linux distribution on your host system which includes the very same Python version. This could be for example Ubuntu 22.04 LTS for Python 3.10.

Best Practices for Developing Custom EVerest Modules¶

It is recommended to develop your custom software on your local PC Linux environment, using a compiler, debugger, and tools that you are familiar with. Using tools like autotools, cmake, and pkg-config in your own projects will make the integration and cross-compiling process much easier.

A native installation of EVerest on your host system is also recommended, as it includes simulation examples that allow you to start and test EVerest locally on your machine. If the software you developed does not depend on specific hardware components, it is also possible to test your EVerest module natively on the local machine before it is executed on the target hardware platform. Please refer to Simulating EVerest for more information on how to start your EVerest module in the SiL environment.

If you are starting your program from scratch and considering additional libraries to support specific functionality, first review the libraries already required by EVerest or included in the firmware’s Linux distribution. Minimizing dependencies helps reduce the size of the firmware update image, resulting in faster transfer and flashing times during board updates.

When developing your software component, you need to decide how it will interact with the EVerest stack. You can either implement your software as an EVerest module and use the internal interfaces to communicate with the stack, or you can use an external API to interact with the EVerest stack. The latter approach uses EVerest modules which provide an external API. Currently, there are two options available:

Using the RpcApi module to connect an energy management system via JSON-RPC over WebSocket.
Using the API module to connect an energy management system via MQTT. Please note that this module is deprecated and will be removed in one of the next releases.

The advantage of not having to deal with the internal structures of EVerest (e.g. setting up an EVerest development environment) and using a more stable API comes with the disadvantage of being limited to the signals and parameters of the external interface. If you want to access additional internal parameters, you usually have to develop your own EVerest module.

Note

EVerest now includes BringUp modules and a dedicated Bring-up guide for a minimal setup that can be used to start and test custom EVerest modules without requiring the full target system. These modules provide mockup implementations for selected interfaces and can be configured via the GUI. This makes it possible to execute interface commands against your own module and verify its behavior during development.

To use this approach, create a minimal EVerest configuration that includes your module together with the required BringUp modules. To test this setup on the target device, both the BringUp modules and the custom module under test need to be cross-compiled for the target platform.

The process of creating your own EVerest modules is described in the EVerest documentation: How to: Develop New Modules.

Note

It is also possible to create your module directly in the EVerest repository in the modules directory. This is only recommended if you either want to work on a fork of the EVerest repository or if you want to contribute the module to the main EVerest project. In general, it is recommended to create your own module in a separate EVerest module repository as described in the EVerest documentation.

Once your own modules work as expected on your local machine, or if this is not possible e.g. due to hardware dependencies, then it is time to cross-compile it for the target platform and test it within the target environment. The cross-compile procedure is described in Cross-compiling EVerest Modules.

In a later development stage, all your custom modules need/should be included into your custom firmware update image so that it is included during firmware updates. This is described in Creating Customized Firmware Images with Yocto.

The following flowchart summarizes the best practices for developing custom EVerest modules:

Cross-compiling EVerest Modules¶

Cross-compilation is the fastest and most convenient way to test your own modules directly on the target system during development. The cross-compiled project is transferred directly via SFTP to the charge controller and then tested in the target environment.

The following steps describe how to cross-compile a module for the Charge Control C platform.

On an Ubuntu or Debian-based Linux distribution, install the required development tools and cross-compilers for the Charge Control C platform:

sudo apt install build-essential squashfs-tools rauc
sudo apt install gcc-arm-linux-gnueabihf g++-arm-linux-gnueabihf binutils-arm-linux-gnueabihf

Download chargebyte’s digital certificate keyring and use it to extract the root filesystem from the firmware image. The latest firmware images can be found in the “Releases” section of chargebyte’s Yocto EVerest meta-layer.
```
rauc extract --keyring=./path/to/chargebyte-keyring.pem ./path/to/downloaded-firmware.image bundle-staging
```
Note

Alternatively, if the above command does not work (e.g. RAUC is not available on your host system), you can use the following command:
unsquashfs -d bundle-staging ./path/to/downloaded-firmware.image
However, this will not verify the signature of the firmware image.
The extracted update image file contains an ext4 root filesystem image tailored for the Charge Control C platform. This must be mounted as a loop device. A read-only mount is recommended and sufficient here:
```
sudo mkdir -p /srv/chargebyte/everest/everest-devel-image-mount-tarragon
sudo mount bundle-staging/core-image-minimal-tarragon.ext4 /srv/chargebyte/everest/everest-devel-image-mount-tarragon -o ro
```
We recommend making this mount persistent using your host system’s /etc/fstab or similar tools, but this is out of scope for this documentation.

Note

Keep in mind that such a firmware image is specific to a target platform, the Yocto version used, and the EVerest release. Adapt these instructions accordingly if you need to develop for different versions in parallel.
Create a new toolchain directory in your everest-workspace directory (next to the everest directory) and create a file named toolchain.cmake:
```
cd everest-workspace
mkdir toolchain
cd toolchain
touch toolchain.cmake
```

Save the following content in the toolchain.cmake file:

set(CMAKE_SYSTEM_NAME Linux)

if(EXISTS ${CMAKE_SYSROOT} AND IS_DIRECTORY ${CMAKE_SYSROOT})
  execute_process(COMMAND file ${CMAKE_SYSROOT}/bin/bash.bash
		  OUTPUT_VARIABLE TARGET_BASH_OUTPUT)
  if(TARGET_BASH_OUTPUT MATCHES aarch64)
    message(STATUS "Setting architecture to aarch64")
    set(CMAKE_SYSTEM_PROCESSOR aarch64)
    set(CMAKE_C_COMPILER /usr/bin/aarch64-linux-gnu-gcc)
    set(CMAKE_CXX_COMPILER /usr/bin/aarch64-linux-gnu-g++)
    set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} " CACHE STRING "" FORCE )
    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} " CACHE STRING "" FORCE )
  else()
    message(STATUS "Setting architecture to arm")
    set(CMAKE_SYSTEM_PROCESSOR arm)
    set(CMAKE_C_COMPILER /usr/bin/arm-linux-gnueabihf-gcc)
    set(CMAKE_CXX_COMPILER /usr/bin/arm-linux-gnueabihf-g++)
    set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -Wno-psabi" CACHE STRING "" FORCE )
    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-psabi" CACHE STRING "" FORCE )
  endif()
else()
  message(FATAL_ERROR "ERROR: SYSROOT '${CMAKE_SYSROOT}' not found!!!")
endif()

 if(CMAKE_BUILD_TYPE MATCHES Debug)
    # Debug flags
    message("Enabling Debug build")
    set(CMAKE_CXX_FLAGS_DEBUG "-g")
else()
    # Enable compiler optimization flags
    set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -Os")
    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Os")

    # Strip debug symbols
    set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -s")
endif()

set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -L${CMAKE_SYSROOT}/usr/lib")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -L${CMAKE_SYSROOT}/usr/lib")

set(ENV{PKG_CONFIG_PATH} "${CMAKE_SYSROOT}/usr/lib/pkgconfig:$ENV{PKG_CONFIG_PATH}")

set(CMAKE_CXX_STANDARD_LIBRARIES "${CMAKE_SYSROOT}/usr/lib/libstdc++.so")

set(NODEJS_INCLUDE_DIR /usr/include/node) # make sure that nodejs is installed. If not, sudo apt-get install nodejs-dev

set(PYTHON_INCLUDE_DIRS "${CMAKE_SYSROOT}/usr/include/python3.10")
set(PYTHON_LIBRARIES "${CMAKE_SYSROOT}/usr/lib/libpython3.10.so")

set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)

The resulting directory structure should now look like this:

everest-workspace/
|── {MyEVerestModuleRepo}
├── everest
└── toolchain
    └── toolchain.cmake

Create a new build-tarragon directory in the MyEVerestModuleRepo directory (i.e. within your own EVerest module project directory or in everest if you want to build EVerest modules or your module is part of the EVerest repository):
```
cd {MyEVerestModuleRepo}
mkdir build-tarragon
cd build-tarragon
```

Run the following command inside the build-tarragon directory to configure the build:

CMAKE_INSTALL_PATH_DEFINES=" \
    -DCMAKE_INSTALL_PREFIX:PATH=/usr \
    -DCMAKE_INSTALL_BINDIR:PATH=/usr/bin \
    -DCMAKE_INSTALL_SBINDIR:PATH=/usr/sbin \
    -DCMAKE_INSTALL_LIBEXECDIR:PATH=/usr/libexec  \
    -DCMAKE_INSTALL_SYSCONFDIR:PATH=/etc \
    -DCMAKE_INSTALL_SHAREDSTATEDIR:PATH=/var/share \
    -DCMAKE_INSTALL_LOCALSTATEDIR:PATH=/var \
    -DCMAKE_INSTALL_LIBDIR:PATH=/usr/lib \
"
export CMAKE_INSTALL_PATH_DEFINES

cmake \
    $CMAKE_INSTALL_PATH_DEFINES \
    -DCMAKE_TOOLCHAIN_FILE=../../toolchain/toolchain.cmake \
    -DCMAKE_SYSROOT=/srv/chargebyte/everest/everest-devel-image-mount-tarragon \
    ..

After the configuration completes successfully, cross-compile the module using make:
```
make {MyEVerestModule} -j$(nproc)
```
This builds only your own module and all dependencies. If you encounter build problems, reduce the number of CPUs used to a single one using -j1. This makes it easier to locate the problem in the output log. Repeat this command if you want to build multiple modules.
After completing the previous step, you can build and install the remaining EVerest modules:
```
make install DESTDIR=target -j$(nproc)
```

If the build was successful, a target subdirectory is created within build-tarragon with the cross-compiled binaries and the module manifest files. Check whether the following directory structure exists:

target/
└── usr
    └── libexec
        └── everest
            └── modules
                └── {MyEVerestModule}
                    ├── {MyEVerestModule} (binary)
                    └── manifest.yaml (manifest file)

Verify that the resulting binaries were compiled for the Charge Control C platform:

file target/usr/libexec/everest/modules/{MyEVerestModule}/{MyEVerestModule}

The output should look similar to this:

usr/libexec/everest/modules/{MyEVerestModule}/{MyEVerestModule}: ELF 32-bit LSB pie executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, BuildID[sha1]=cf7fc6ab427435ba5e30254eb234e3dba5fad685, for GNU/Linux 3.2.0, stripped

The resulting binary and manifest can be found in the target/usr/libexec/everest/modules/{MyEVerestModule} directory. If you want to test the module on the target system, you can copy the module directory using scp or rsync.

You cannot replace the binary on the target system while it is running. First, stop the EVerest system on the target:
```
systemctl stop everest
```
Back on your host system, you can now transfer the files:
```
scp -r target/usr/libexec/everest/modules/{MyEVerestModule} root@<ip_address>:/usr/libexec/everest/modules/
```
Then restart the EVerest system on the target system:
```
systemctl start everest
```

Note

Using this approach, it is also possible to transfer not only your own module(s), but also the complete content of the target directory to the target system. This is helpful if you want to test changes that also affect EVerest libraries or if you want to test a newer EVerest version.

Creating Customized Firmware Images with Yocto¶

As already mentioned in the Firmware chapter, chargebyte’s firmware images are built using the Yocto Project build system. This chapter describes how to set up the Yocto build environment for chargebyte’s firmware images and how to create a customized firmware image by adding a new EVerest module based on the chargebyte Yocto layer.

Setting up the Yocto Build Environment¶

The Yocto build environment is based on the chargebyte BSP (Board Support Package). Please refer to our Board Support Package documentation in our public GitHub repository: https://github.com/chargebyte/chargebyte-bsp for detailed instructions.

Install the required packages for Yocto on a Linux machine or virtual machine. (Note: We usually set up the Yocto build environment on an Ubuntu 20.04 or later Linux distribution.)
Install repo to help set up your Yocto environment. The repo utility makes it easy to reference several Git repositories within a top-level project, which you can then clone to your local machine all at once.
```
mkdir ~/bin
curl http://commondatastorage.googleapis.com/git-repo-downloads/repo > ~/bin/repo
chmod a+x ~/bin/repo
```
Ensure that ~/bin is added to your PATH variable (On Ubuntu, this directory is usually added automatically).
```
echo 'export PATH="$PATH":~/bin' >> ~/.bashrc
```
Use the repo tool to check out the Yocto layers required to build the firmware image. This requires a manifest file that contains information about the repositories for the necessary Yocto layers and the specific branches to be checked out. The manifest file can be found in a repository called “chargebyte-bsp”. (Note: chargebyte’s Yocto build environment is currently based on ‘kirkstone’ – an LTS release of the Yocto Project, which ensures long-term support and stability for embedded Linux systems.)
```
mkdir yocto
cd yocto
repo init -u https://github.com/chargebyte/chargebyte-bsp -b kirkstone-everest
repo sync
```
It may take a few minutes to download all repositories using the repo sync command. After execution, you should see three folders in the created yocto directory:
1. source: Contains all the repositories representing the layers.
2. chargebyte-bsp: A clone of the ‘chargebyte-bsp’ repository, including the manifest and configuration files.
3. build: Initially links to the conf folder in chargebyte-bsp.

Follow the documentation in the ‘chargebyte-bsp’ repository to build a firmware image based on the specific board support package (BSP). Currently supported hardware platforms are Tarragon (Charge Control C) and Charge SOM.

The next step is to integrate a new EVerest module and build a custom image that incorporates it.

Integrating a Custom EVerest Module¶

The EVerest documentation explains the module concept in detail and provides a guide on how to develop a new EVerest module.

This section focuses on integrating the module into the Yocto build system.

To integrate custom EVerest modules into the Yocto build system, you can either create a new Yocto layer or extend an existing one. This section assumes that a new layer has been created and added to the BBLAYERS variable in the build/conf/bblayers.conf file.

A recipe file is required to build the module. A recipe is a file with the extension .bb and contains information such as the source code location, dependencies, and build instructions. Refer to the Yocto documentation for a guide on writing recipe files. Let’s assume the new recipe is named my-module.bb. It might look like this:

SUMMARY = "My Module"
DESCRIPTION = "A new EVerest module"

LICENSE = "APACHE-2.0"
LIC_FILES_CHKSUM = "file://LICENSE;md5=1234567890"

SRC_URI = "git://github.com/my_org/my-module.git;branch=main"
S = "${WORKDIR}/git"

inherit cmake

DEPENDS = "lib1 lib2"

do_install() {
    install -d ${D}${bindir}
    install -m 0755 ${B}/my-module ${D}${bindir}
}

Add the recipe name my-module to the IMAGE_INSTALL variable in the build/conf/local.conf file to include the module in the image.

The module is now integrated into the Yocto build system. The next step is to build the custom image.

Creating a Development Image¶

To build the custom image, follow the section “Building an image” in the “chargebyte-bsp” repository. This process will generate a Linux root filesystem, which can then be either flashed directly, or used to create a firmware image using RAUC.

The resulting custom image should now include your new EVerest module.

Logging and Debugging¶

There are different ways to analyze the EVerest charging software. In the following, different ways are described. In preparation for starting this chapter, it is necessary to be connected to the board via Ethernet. This is also necessary to copy logs and traces from the board to the local PC via SFTP tool like WinSCP or FileZilla.

journalctl Logs¶

As already described in the Starting and Monitoring the Charging Process section, the EVerest framework uses systemd’s journalctl for logging. journalctl is a system service that collects and stores logging data. To view the logs, connect to the charge controller via SSH and run the following command:

journalctl -f -n 100 -u everest

The -f flag is used to follow the logs live, and the -n flag is used to view the last 100 lines only. The -u flag is used to filter the logs for the EVerest service. To stop following the logs, press Ctrl+C.

The logs are stored by the systemd journal service in binary format under /var/log/journal. Accessing these files on the charge controller itself should only be done using the journalctl tool.

But in case, you transferred the files to your host system, you can also use journalctl on your host system:

journalctl -u everest --file /path/to/your/file.journal

As shown in Starting and Monitoring the Charging Process, a log line consists, among other things, of a timestamp, a log level, a module id, a module name, and a message. The log level can be one of the following:

“VERB” (Verbose)

“DEBG” (Debug)

“INFO” (Information)

“WARN” (Warning)

“ERRO” (Error)

“CRIT” (Critical)

The log level can be configured for each EVerest module separately in the logging configuration file. The logging configuration file is located at /etc/everest/default_logging.cfg. The filter can be configured using the log level and the module identifier as defined in the EVerest configuration. Here is an example to configure the message filter to change the log level of the EvseManager module to DEBG:

# The filter expression consists of two parts:
# 1. Logs with severity level INFO or higher (affects all EVerest modules).
# 2. Logs from the EvseManager with module id "connector" with severity level DEBG or higher.
Filter="(%Severity% >= INFO) or (%Process% contains connector and %Severity% >= DEBG)"

PCAP Traces¶

pcap traces can be used to debug and analyze the high-level communication (HLC) between SECC and EVCC. In general, there are two ways to capture pcap traces:

Capture via tcpdump command
```
tcpdump -i eth1 -w /srv/charge_com_01.pcap
```
This command captures all packets on the eth1 (PLC) interface and writes them to a pcap file. Stop the capture by pressing Ctrl+C.

Note

Please stop the trace before copying the file to a local machine. The file may be incomplete if copied while the tcpdump is still running.
Using the EVerest PacketSniffer module

The PacketSniffer module is part of the main EVerest repository and can be used to capture pcap traces automatically. Please refer to the module manifest to see how to configure the PacketSniffer module.

Note

The PacketSniffer module should not be part of a production configuration. Depending on the configured capture behavior, it can continuously write trace data and fill up the available storage.

pcap traces with V2G communication can be analyzed using Wireshark with a V2G plugin like the one from dSPACE.

Note

TLS encrypted communication cannot be analyzed with Wireshark without the TLS session key. Please look in the module manifest which is responsible for the TLS handshake and check if there is an option to export the TLS session key. E.g. the IsoMux module is able to export the TLS session key.

CAN Traces¶

CAN traces can be used to debug and analyze the communication between the charge controller and external peripheral like power supply. The following command starts a CAN trace with absolute timestamps:

candump -tA -l can0

It captures all CAN packets on the can0 interface and writes them to an ASCII file in the current working directory. Stop the capture by pressing Ctrl+C.

Session Logging of the EVSEManager¶

The EVerest EVSEManager can generate session log files with information about the current session state, configured CP state and duty cycle. It can also decode and log the content of the V2G messages. Please refer to the module manifest to see how to configure the EVSEManager module to activate session logging.

Note

The session logging is pretty performance intensive and should only be used for debugging purposes. It should not be enabled in production configurations, because the generated log files can grow quickly and fill up the available storage.

MQTT Logs¶

It is also possible to observe the internal MQTT communication between the EVerest modules. When developing new modules, this can be helpful to better understand the internal processes regarding the use of the interfaces. We recommend using the MQTT Explorer tool to monitor and analyze the MQTT communication. It is also possible to use the following command to monitor the MQTT communication, but it is not easy to read:

mosquitto_sub -F "[@H:@M:@S.@N] %t %p" -t 'everest/#' -t 'everest_api/#' > /srv/charge_com_mqtt_01.log

Summary¶

There are several ways to debug and analyze the EVerest framework. Which way to choose depends on the specific use case and the information needed. The most common way is using journalctl logs. If it is necessary to analyze the high-level communication between SECC and EVCC, pcap traces are the best choice. If it is not clear how the internal communication between the EVerest modules works, the MQTT logs can be helpful.

Development¶

Preparation Checklist for Development¶

Best Practices for Developing Custom EVerest Modules¶

Cross-compiling EVerest Modules¶

Creating Customized Firmware Images with Yocto¶

Setting up the Yocto Build Environment¶

Integrating a Custom EVerest Module¶

Creating a Development Image¶

Logging and Debugging¶

journalctl Logs¶

PCAP Traces¶

CAN Traces¶

Session Logging of the EVSEManager¶

MQTT Logs¶

Summary¶

Table of Contents

Previous topic

Next topic