Development¶

As mentioned in the section “Programming”, customers can create their own applications and integrate them into a custom firmware image. This section will guide you through the process of creating a custom EVerest module and integrating it into an image. This is done by either using the Yocto build system or cross-compiling the application for the chargebyte hardware platform.

Setting up Yocto Build Environment¶

Install the required packages for Yocto on a Linux machine / virtual machine. (Note: We normally set up the Yocto build environment on an Ubuntu 20.04 or later Linux distribution.)
Install repo to help getting your Yocto environment ready. The repo utility makes it easy to reference several Git repositories within a top-level project, which you can then clone to your local machine all at once.
```
mkdir ~/bin
curl http://commondatastorage.googleapis.com/git-repo-downloads/repo > ~/bin/repo
chmod a+x ~/bin/repo
```
You need to also make sure that ~/bin is added to your PATH variable (Usually the directory is added automatically in Ubuntu).
```
echo 'export PATH="$PATH":~/bin' >> ~/.bashrc
```
The repo tool should be used to checkout the Yocto layers needed to build the firmware image. This requires a manifest file containing information about the repositories for the necessary Yocto layers and the specific branches to be checked out. The manifest file can be found in a repository called “chargebyte-bsp”. (Note: chargebyte’s Yocto build environment is currently based on ‘Kirkstone’ - a LTS release of the Yocto Project).
```
mkdir yocto
cd yocto
repo init -u https://github.com/chargebyte/chargebyte-bsp -b kirkstone-everest
repo sync
```
It should take a couple of minutes to download all the repositories using the command repo sync. After the command is executed, you should be able to find three folders in the created yocto directory:
1. source: Where all the repositories representing the layers are cloned.
2. chargebyte-bsp: A clone of the ‘chargebyte-bsp’ repository containing the manifest file and configurations folder.
3. build: Initially holds a link to the conf folder in chargebyte-bsp.

Follow the documentation in the ‘chargebyte-bsp’ repository to build a firmware image based on the hardware platform specific board support package (BSP). Currently supported are the hardware platforms Tarragon (Charge Control C) and Charge SOM.

This will include EVerest and chargebyte’s hardware abstraction layer (HAL).

The next step in this chapter is to write a new EVerest module and build a custom image that incorporates this new module.

Adding a Custom EVerest Module¶

The EVerest documentation explains the modules in detail and their configurations, and includes a guide on how to develop an new EVerest module.

This section will focus on integrating the module into the Yocto build system.

In order to integrate custom EVerest modules into the Yocto build system, you can either create a new Yocto layer or extend an existing one. This section will assume that a new layer has been created and added to the BBLAYERS variable in the build/conf/bblayers.conf file.

A recipe file is needed to build the module. A recipe is a file with the extension .bb and contains information about the module, such as the source code location, dependencies, and how to build it. The Yocto documentation provides a guide on how to write a recipe file. Let’s assume that the new recipe is called my-module.bb. It should look something like this:

SUMMARY = "My Module"
DESCRIPTION = "A new EVerest module"

LICENSE = "APACHE-2.0"
LIC_FILES_CHKSUM = "file://LICENSE;md5=1234567890"

SRC_URI = "git://github.com/my_org/my-module.git;branch=main"
S = "${WORKDIR}/git"

inherit cmake

DEPENDS = "lib1 lib2"

do_install() {
    install -d ${D}${bindir}
    install -m 0755 ${B}/my-module ${D}${bindir}
}

Add the name of the recipe my-module to the IMAGE_INSTALL variable in the build/conf/local.conf file so that the module is included in the image.

The module is now integrated into the Yocto build system. The next step is to build the custom image.

Creating a Development Image¶

In order to build the custom image, follow the section “Building an image” found in the “chargebyte-bsp” repository which produces a Linux root filesystem. This can be either flashed directly, or used to create a firmware image using RAUC.

The custom image should now include the new EVerest module.

Cross-compiling for Charge SOM¶

Another way to integrate custom applications into the firmware image is to cross-compile the application for Charge SOM and include it in the image. A pre-requisite for this is to have the latest firmware image as a developer build. Always keep in mind, if you want to build a new EVerest module it must be compatible to the EVerest release within the firmware. Please have a look at the official EVerest documentation, how to checkout a dedicated EVerest release.

On an Ubuntu or Debian-based Linux distribution, install the cross-compilers for Charge SOM.

sudo apt install build-essential libc6-arm64-cross gcc-aarch64-linux-gnu binutils-arm-linux-gnueabihf gcc-arm-linux-gnueabihf g++-arm-linux-gnueabihf

Download chargebyte’s digital certificate and use it to extract the root filesystem from the firmware image. The digital certificate is the same between Charge SOM and Charge Control C.
```
rauc extract --keyring=<chargebyte_certificate>.crt <shipped_firmware>.image bundle-staging
```
Note
Alternatively, if the above command does not work, you can use the following command:
unsquashfs -d bundle-staging <shipped_firmware>.image
But this will not verify the signature of the firmware image.

Mount the ext4 root filesystem image as a loop device.

sudo mkdir -p /mnt/rootfs
sudo mount bundle-staging/core-image-minimal-chargesom.ext4 /mnt/rootfs

Create a new directory in the folder where the new module was created (my-module) and create a new file called toolchain.cmake. This file is used to set the toolchain for the cross-compilation.
```
cd my-module
mkdir toolchain
cd toolchain
touch toolchain.cmake
```

Store the following lines in the toolchain.cmake file:

set(CMAKE_SYSTEM_NAME Linux)

if(EXISTS ${CMAKE_SYSROOT} AND IS_DIRECTORY ${CMAKE_SYSROOT})
  execute_process(COMMAND file ${CMAKE_SYSROOT}/bin/bash.bash
		  OUTPUT_VARIABLE TARGET_BASH_OUTPUT)
  if(TARGET_BASH_OUTPUT MATCHES aarch64)
    message(STATUS "Setting architecture to aarch64")
    set(CMAKE_SYSTEM_PROCESSOR aarch64)
    set(CMAKE_C_COMPILER /usr/bin/aarch64-linux-gnu-gcc)
    set(CMAKE_CXX_COMPILER /usr/bin/aarch64-linux-gnu-g++)
    set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} " CACHE STRING "" FORCE )
    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} " CACHE STRING "" FORCE )
  else()
    message(STATUS "Setting architecture to arm")
    set(CMAKE_SYSTEM_PROCESSOR arm)
    set(CMAKE_C_COMPILER /usr/bin/arm-linux-gnueabihf-gcc)
    set(CMAKE_CXX_COMPILER /usr/bin/arm-linux-gnueabihf-g++)
    set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -Wno-psabi" CACHE STRING "" FORCE )
    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-psabi" CACHE STRING "" FORCE )
  endif()
else()
  message(FATAL_ERROR "ERROR: SYSROOT '${CMAKE_SYSROOT}' not found!!!")
endif()

 if(CMAKE_BUILD_TYPE MATCHES Debug)
    # Debug flags
    message("Enabling Debug build")
    set(CMAKE_CXX_FLAGS_DEBUG "-g")
else()
    # Enable compiler optimization flags
    set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -Os")
    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Os")

    # Strip debug symbols
    set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -s")
endif()

set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -L${CMAKE_SYSROOT}/usr/lib")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -L${CMAKE_SYSROOT}/usr/lib")

set(ENV{PKG_CONFIG_PATH} "${CMAKE_SYSROOT}/usr/lib/pkgconfig:$ENV{PKG_CONFIG_PATH}")

set(CMAKE_CXX_STANDARD_LIBRARIES "${CMAKE_SYSROOT}/usr/lib/libstdc++.so")

set(NODEJS_INCLUDE_DIR /usr/include/node) # make sure that nodejs is installed. If not, sudo apt-get install nodejs-dev

set(PYTHON_INCLUDE_DIRS "${CMAKE_SYSROOT}/usr/include/python3.10")
set(PYTHON_LIBRARIES "${CMAKE_SYSROOT}/usr/lib/libpython3.10.so")

set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)

Create a new build directory in “my-module” and navigate to it.
```
mkdir build
cd build
```

Run the following command inside to configure the build.

cmake -DCMAKE_TOOLCHAIN_FILE=../toolchain/toolchain.cmake -DCMAKE_SYSROOT=/mnt/rootfs ..

When this ends successfully, start cross-compiling using make:
```
make install -j$(nproc)
```

Test that the resulting binaries are compiled for Charge SOM as a target:

file dist/libexec/everest/modules/MyModule/MyModule

The output should be something like:

dist/libexec/everest/modules/MyModule/MyModule: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV),
dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, BuildID[sha1]=ad2342fdd3b8fb1949fc3e13b77382d3da72f28a, for GNU/Linux 3.7.0, stripped

The resulting binary and manifest file can be copied to the previously mounted root filesystem.
```
cp dist/libexec/everest/modules/MyModule /mnt/rootfs/usr/libexec/everest/modules/
```
umount the loop device.
```
sudo umount /mnt/rootfs
```

Make sure that the customized filesystem is in a clean state.

fsck.ext4 -f bundle-staging/core-image-minimal-chargesom.ext4

Follow the steps under the section Firmware Update Customization and Signing to install your PKI certificate, pack the modified root filesystem image again into the firmware update image, and test the new firmware image.

Logging and Debugging¶

There are different ways to analyze the EVerest charging software. In the following, different ways are described. In preparation for starting this chapter, it is necessary to be connected to the board via Ethernet. This is also necessary to copy logs and traces from the board to the local PC via a SFTP tool like WinSCP or FileZilla.

journalctl logs¶

As already described in the Starting and Monitoring the Charging Process section, the EVerest framework uses journalctl for logging. journalctl is a system service that collects and stores logging data. To view the logs, connect to the charge controller via SSH and run the following command:

journalctl -f -n 100 -u everest

The -f flag is used to follow the logs live, and the -n flag is used to view the last 100 lines. The -u flag is used to filter the logs for the everest service. To stop following the logs, press Ctrl + C.

The logs are stored by the systemd journal service in a binary format in “/var/log/journal”. To open the binary in a human-readable format, use the following command:

journalctl -u everest --file /path/to/your/file.journal

As shown in the Starting and Monitoring the Charging Process section, a log line consists, among other things, of a timestamp, a log level, a module id, a module name, and a message. The log level can be one of the following:

“VERB” (Verbose)

“DEBG” (Debug)

“INFO” (Information)

“WARN” (Warning)

“ERRO” (Error)

“CRIT” (Critical)

The log level can be configured for each EVerest module separately in the logging configuration file. The logging configuration file is located at “/etc/everest/default_logging.cfg”. The filter can be configured using the log level and the module identifier as defined in the EVerest configuration. Here is an example to configure the message filter to change the log level of the EvseManager module to DEBUG:

# The filter expression consists of two parts:
# 1. Logs with severity level INFO or higher (affects all EVerest modules).
# 2. Logs from the EvseManager with module id "connector" with severity level DEBUG or higher.
Filter="(%Severity% >= INFO) or (%Process% contains connector and %Severity% >= DEBG)"

pcap traces¶

pcap traces can be used to debug and analyze the high-level communication (HLC) between SECC and EVCC. In general, there are two ways to capture pcap traces:

Capture via tcpdump command
tcpdump -i eth1 -w /srv/charge_com_01.pcap
This command captures all packets on the eth1 (PLC) interface and writes them to a pcap file. Stop the capture by pressing Ctrl + C.

Note

Please stop the trace before copying the file to a local machine. The file may be incomplete if copied while the tcpdump is still running.
Using the EVerest “PacketSniffer” module

The PacketSniffer module is part of everest-core and can be used to capture pcap traces automatically. Please look into the module manifest to see how to configure the PacketSniffer module.

pcap traces with V2G communication can be analyzed using Wireshark with a V2G plugin like from dSPACE.

Note

TLS encrypted communication cannot be analyzed with Wireshark without the TLS session key. Please look in the module manifest which is responsible for the TLS handshake and check if there is an option to export the TLS session key. E.g. the IsoMux module is able to export the TLS session key.

Session logging of the EVSEManager¶

The EVerest “EVSEManager” can generate session log files with information about the current session state, configured CP state and duty cycle. It can also decode and log the content of the V2G messages. Please look into the module manifest to see how to configure the EVSEManager module to activate session logging.

Note

The session logging is pretty performance intensive and should only be used for debugging purposes.

MQTT logs¶

It is also possible to observe the internal MQTT communication between the EVerest modules. When developing new modules, this can be helpful to better understand the internal processes regarding the use of the interfaces. We recommend using the MQTT Explorer tool to monitor and analyze the MQTT communication. It is also possible to use the following command to monitor the MQTT communication, but this is not so easy to read:

mosquitto_sub -F "[@H:@M:@S.@N] %t %p" -t 'everest/#' -t 'everest_api/#' > /srv/charge_com_mqtt_01.log

Summary¶

There are several ways to debug and analyze the EVerest framework. Which way to choose depends on the specific use case and the information needed. The most common way is using journalctl logs. If it is necessary to analyze the high-level communication between SECC and EVCC, pcap traces are the best choice. If it is not clear how the internal communication between the EVerest modules works, the MQTT logs can be helpful.