Zeroincombenze® continuous testing for odoo
This package is an plug-in of zerobug package and aim to easily create odoo tests.
It replaces OCA MQT with some nice additional features.
z0bug_odoo is built on follow concepts:
Odoo version independent; it can test Odoo from 6.1 until 16.0
It is designed to run in local environment too, using local travis emulator
It can run with full or reduced set of pylint tests
Test using ready-made database records
Quality Check Id
travis ci support
The goal of z0bug_odoo is to provide helpers to ensure the quality of Odoo addons. The code was forked from OCA MQT but some improvements were added. This package and OCA MQT differ by:
z0bug_odoo can also test Odoo 6.1 and 7.0 where OCA MQT fails with these versions
z0bug_odoo is designed to execute some debug statements, mainly in local environment
z0bug_odoo has more options to run with reduced set of lint tests
OCA MQT is the only component to build environment and test Odoo while z0bug_odoo is part of Zeroincombenze® tools
As per prior rule, building test environment is made by vem, clodoo and lisa. These commands can also build a complete Odoo environment out of the box
To make a complete test on TravisCI your project following 3 files are required:
.travis.yml
requirements.txt
oca_dependencies.txt
File .travis.yml
In order to setup TravisCI continuous integration for your project, just copy the content of the sample_files to your project’s root directory.
Then execute the command:
make_travis_conf <TOOLS_PATH>/travis_emulator/template_travis.yml .travis.yml
You can check travis syntax with the lint checker of travis, if available.
Notice: if you do not use travisCi web site, you can avoid to set .travis.yml file. Local travis emulator and z0bug_odoo create local .travis.yml dinamically.
Odoo test integration
Current Odoo project version is declared by VERSION variable. If your Odoo module must be tested against Odoo core, you can test specific github repository by ODOO_REPO variable. You can test against:
odoo/odoo
OCA/OCB
zeroincombenze/OCB
librerp/OCB
You can test against specific Odoo core version with ODOO_BRANCH variable if differs from your project version:
# Odoo Branch 16.0
- VERSION="16.0" ODOO_REPO="odoo/odoo"
# Pull request odoo/odoo#143
- VERSION="pull/143" ODOO_REPO="OCA/OCB"
# Branch saas-17
- ODOO_REPO="odoo/odoo" ODOO_BRANCH="saas-17"
OCB / core test
Zeroincombenze® OCB uses submodules. When test is starting, travis-ci upgrades repository and submodules. To avoid submodules upgrade use this directive compatible with OCA MQT:
- git:
submodules: false
z0bg_odoo set security environment. You do not need to add any security statements. You can avoid the following OCA MQT directive:
- pip install urllib3[secure] --upgrade; true
z0bg_odoo does some code upgrade. You can avoid following directive in ODOO_TEST_SELECT=”APPLICATIONS”:
- sed -i "s/self.url_open(url)/self.url_open(url, timeout=100)/g" ${TRAVIS_BUILD_DIR}/addons/website/tests/test_crawl.py;
You can avoid following directive in ODOO_TEST_SELECT=”LOCALIZATION”:
- sed -i "/'_auto_install_l10n'/d" ${TRAVIS_BUILD_DIR}/addons/account/__manifest__.py
Python version
Odoo version from 6.1 to 10.0 are tested with python 2.7 From Odoo 11.0, python3 is used. You can test against 3.5, 3.6, 3.7 and 3.8 python versions. Currently, python 3.8 is not yet supported. This is the declaration:
python:
- "3.5"
- "3.6"
- "3.7"
- "3.8"
Notice: python 3.5 support is ended on 2020 and 3,6 is ended on 2021. Python 3.8 is no yet full supported by Odoo (2021), so use python 3.7
Deployment and setup environment
In order to deploy test environment and setup code you have to declare some .travis.yml directives divides in following 3 parts:
Linux packages needed
PYPI packages
Odoo repositories dependencies
Linux packages must be declared in <addons/apt> section of .travis.yml using Ubuntu namespace. If you run test in local environment, travis emulator automatically translate Ubuntu names into your local distro names, if necessary. See travis emulator guide for furthermore info.
The PYPI packages, installable by PIP are declared in standard PIP way, using requirements.txt file.
If your project depends on other Odoo Github repositories like OCA, create a file called oca_dependencies.txt at the root of your project and list the dependencies there. One per line like so:
project_name optional_repository_url optional_branch_name
During testbed setup, z0bug_odoo will automatically download and place these repositories accordingly into the addon path. Note on addons path ordering: they will be placed after your own repo, but before the odoo core repo.
If missed optional_repository_url, the repository is searched for repository with the same owner of tested project. Please note this behaviour differs from OCA MQT. OCA MQT always loads OCA repository while z0bug_odoo searches for current owner repository. So you will test both with z0bug_ood and both OCA MQT, always insert the full repository URL.
Test execution
Tests run by travis_run_test command. The script is deployed in _travis directory of zerobug package. Command have to be in <script> section of .travis.yml file:
script:
- travis_run_tests
Isolated pylint+flake8 checks
If you want to make a build for these checks, you can add a line on the <env> section of the .travis.yml file with this content:
- VERSION="12.0" LINT_CHECK="1"
To avoid making again these checks on other builds, you have to add LINT_CHECK=”0” variable on the line:
- VERSION="12.0" ODOO_REPO="odoo/odoo" LINT_CHECK="0"
You can superset above options in local travis emulator.
Reduced set of lint check
You can execute reduced set of check, in order to gradually evolve your code quality when you meet too many errors.
To enable reduced set of check add one of follow lines:
- LINT_CHECK="1" LINT_CHECK_LEVEL="MINIMAL"
- LINT_CHECK="1" LINT_CHECK_LEVEL="REDUCED"
- LINT_CHECK="1" LINT_CHECK_LEVEL="AVERAGE"
- LINT_CHECK="1" LINT_CHECK_LEVEL="NEARBY"
- LINT_CHECK="1" LINT_CHECK_LEVEL="OCA"
Odoo core has internal pylint test that checks for all modules even the dependecies. So if some dependecies module does not meet this test, then the full travis test fails without testing the target repository.
Please, add test_lint to EXCLUDE variable to avoid this fail-over. See below for furthermore informations.
Look at follow table to understand which tests are disabled at specific level:
FLAKE8 (see http://flake8.pycqa.org/en/latest/user/error-codes.html for deatils)
Test |
MINIMAL |
REDUCED |
AVERAGE |
NEARBY |
OCA |
Note |
E117 |
|
|
|
over-indented |
||
E121 |
|
|
|
|||
E123 |
|
|
|
Closing bracket does not match indentation of opening bracket’s line |
||
E124 |
|
|
|
|||
E126 |
|
|
|
|||
E127 |
|
|
|
|||
E128 |
|
|
|
|||
E131 |
|
|
|
|||
E133 |
|
|
|
|||
E201 |
|
|
|
|||
E202 |
|
|
|
|||
E203 |
|
|
|
|||
E211 |
|
|
|
|||
E221 |
|
|
|
|||
E222 |
|
|
|
|||
E225 |
|
|
|
|||
E226 |
|
|
|
|||
E231 |
|
|
|
|||
E241 |
|
|
|
|||
E242 |
|
|
|
|||
E251 |
|
|
|
|||
E261 |
|
|
|
|||
E262 |
|
|
|
|||
E265 |
|
|
|
|||
E266 |
|
|
|
|||
E271 |
|
|
|
|||
E272 |
|
|
|
|||
W291 |
|
|
|
|||
W292 |
|
|
|
|||
W293 |
|
|
|
|||
E301 |
|
|
|
|||
E302 |
|
|
|
No __init__.py |
||
E303 |
|
|
|
|||
E305 |
|
|
|
|||
W391 |
|
|
|
blank line at end of file |
||
F401 |
|
|
|
module imported but unused |
||
E501 |
|
|
|
|||
E502 |
|
|
|
|||
W503 |
|
|
|
No __init__.py |
||
W504 |
|
|
|
No __init__.py |
||
F601 |
|
|
|
dictionary key name repeated with different values |
||
E701 |
|
|
|
multiple statements on one line (colon) |
||
E722 |
|
|
|
do not use bare except |
||
F811 |
|
|
|
redefinition of unused name from line N (No __init__.py) |
||
F841 |
|
|
|
PYLINT (see http://pylint-messages.wikidot.com/all-codes for details)
Test |
MINIMAL |
REDUCED |
AVERAGE |
NEARBY |
OCA |
Notes |
W0101 |
|
|
|
|||
W0312 |
|
|
|
|||
W0403 |
|
|
|
relative-import |
||
W1401 |
|
|
|
anomalous-backslash-in-string |
||
E7901 |
|
|
|
|||
C7902 |
|
|
|
missing-readme |
||
W7903 |
|
|
|
javascript-lint |
||
W7908 |
|
|
|
missing-newline-extrafiles |
||
W7909 |
|
|
|
redundant-modulename-xml |
||
W7910 |
|
|
|
wrong-tabs-instead-of-spaces |
||
W7930 |
|
|
|
|||
W7935 |
|
|
|
missing-import-error |
||
W7940 |
|
|
|
dangerous-view-replace-wo-priority |
||
W7950 |
|
|
|
odoo-addons-relative-import |
||
E8102 |
|
|
|
invalid-commit |
||
C8103 |
|
|
|
|||
W8103 |
|
|
|
translation-field |
||
C8104 |
|
|
|
|||
W8104 |
|
|
|
api-one-deprecated |
||
C8105 |
|
|
|
|||
C8108 |
|
|
|
method-compute |
||
R8110 |
|
|
|
old-api7-method-defined |
||
W8202 |
|
|
|
use-vim-comment |
||
N/A |
|
|
|
sql-injection |
||
N/A |
|
|
|
duplicate-id-csv |
||
N/A |
|
|
|
create-user-wo-reset-password |
||
N/A |
|
|
|
dangerous-view-replace-wo-priority |
||
N/A |
|
|
|
translation-required |
||
N/A |
|
|
|
duplicate-xml-record-id |
||
N/A |
|
|
|
no-utf8-coding-comment |
||
N/A |
|
|
|
attribute-deprecated |
||
N/A |
|
|
|
consider-merging-classes-inherited |
Disable some pylint and/or flake8 checks
You can disable some specific test or some file from lint checks.
To disable flake8 checks on specific file you can add following line at the beginning of python file:
# flake8: noqa
To disable pylint checks on specific file you can add following line at the beginning of python file:
# pylint: skip-file
To disable both flake8 and pylint checks on specific file you can add following line at the beginning of python file:
# flake8: noqa - pylint: skip-file
To disable pylint checks on specific XML file you can add following line in XML file after xml declaration:
<!-- pylint:disable=deprecated-data-xml-node -->
You can disable specific flake8 check in some source part of python file adding a comment at the same statement to disable check. Here an example to disable sql error (notice comment must be at beginning of the statement):
from builtins import * # noqa: F403
If you have to disable more than one error you can add following declaration:
from builtins import * # noqa
You can also disable specific pylint check in some source part of python file adding a comment at the same statement to disable check. Here an example to disable sql error (notice comment must be at beginning of the statement):
self._cr.execute() # pylint: disable=E8103
Disable unit test
If you want to make a build without tests, you can use the following directive: TEST_ENABLE=”0”
You will simply get the databases with packages installed, but without running any tests.
Reduced set of unit test
Odoo modules may fail in Travis CI or in local environment. Currently Odoo OCB core tests fail; we are investigating for the causes. However you can use a simple workaround, disabling some test. Currently tests fail are:
test_impex
test_ir_actions
test_lint
test_main_flows
test_search
test_user_has_group
Example:
- export EXCLUDE=test_impex,test_ir_actions,test_lint,test_main_flows,test_search,test_user_has_group
- TESTS="1" ODOO_TEST_SELECT="ALL"
- TESTS="1" ODOO_TEST_SELECT="NO-CORE"
- ....
You can set parameter local GBL_EXCLUDE to disable these test for all repositories. You will be warned that local GBL_EXCLUDE has only effect for local emulation. To avoid these test on web travis-ci you have to set EXCLUDE value in .travis.yml file.
Look at follow table to understand which set of tests are enabled or disabled:
statement |
application |
module l10n_* |
odoo/addons |
addons + dependenci |
ALL |
|
|
|
|
APPLICATIONS |
|
|
|
Only if application |
LOCALIZATION |
|
|
|
Only module l10n_* |
CORE |
|
|
|
|
NO-APPLICATION |
|
|
|
No if application |
NO-LOCALIZATION |
|
|
|
No if module l10n_* |
NO-CORE |
|
|
|
|
Dependencies test
Since late Summer 2021, z0bug_odoo checks for dependencies. This test is a sub test of unit test. This is the directive:
- TESTS="1" TEST_DEPENDENCIES="1"
Module unit tests
z0bug_odoo is also capable to test each module individually. The intention is to check if all dependencies are correctly defined. Activate it through the UNIT_TEST directive. An additional line should be added to the env: section, similar to this one:
- VERSION="12.0" UNIT_TEST="1"
Automatic module translation
Since late Summer 2021, z0bug_odoo activate automatic module translation after test ended with success. This is the directive:
- VERSION="12.0" ODOO_TNLBOT="1"
This feature is still experimental.
Names used for the test databases
z0bug_odoo has a nice feature of organizing your testing databases. You might want to do that if you want to double them up as staging DBs or if you want to work with an advanced set of templates in order to speed up your CI pipeline. Just specify at will:
MQT_TEMPLATE_DB=’odoo_template’ MQT_TEST_DB=’odoo_test’.
In your local travis you can declare the default value but these values are not applied in web TravisCi web site.
Database user is the current username. This behavior works both in local test both in TravisCi web site. However, sometimes, local user and db username can be different. You can set the default value in travis emulator.
Coveralls/Codecov configuration file
Coveralls and Codecov services provide information on the test coverage of your modules. Currently both configurations are automatic (check default configuration here. So, as of today, you don’t need to include a .coveragerc into the repository, If you do it, it will be simply ignored.
Other configurations
You can highly customize you test: look at below table.
variable |
default value |
meaning |
CHROME_TEST |
Set value to 1 to use chrome client to test |
|
DATA_DIR |
~/data_dir |
Odoo data directory (data_dir in config file) |
EXCLUDE |
Modules to exclude from test |
|
INCLUDE |
Modules to test (all |
|
INSTALL_OPTIONS |
Options passed to odoo-bin/openerp-server to install modules |
|
MQT_DBSUER |
$USER |
Database username |
MQT_TEMPLATE_DB |
template_odoo |
Read above |
MQT_TEST_DB |
test_odoo |
Read above |
NPM_CONFIG_PREFIX |
$HOME/.npm-global |
N/D |
ODOO_COMMIT_TEST |
0 |
Test result will be committed; require specific code at tear_off function |
ODOO_REPO |
odoo/odoo |
OCB repository against test repository |
ODOO_SETUPS |
__manifest__.py __openerp__.py __odoo__.py __terp__.py |
Names of Odoo manifest files |
ODOO_TEST_SELECT |
ALL |
Read above |
ODOO_TNLBOT |
0 |
Read above |
OPTIONS |
Options passed to odoo-bin/openerp-server to execute tests |
|
PHANTOMJS_VERSION |
Version of PhantomJS |
|
PS_TXT_COLOR |
0;97;40 |
N/D |
PS_RUN_COLOR |
1;37;44 |
N/D |
PS_NOP_COLOR |
34;107 |
N/D |
PS_HDR1_COLOR |
97;42 |
N/D |
PS_HDR2_COLOR |
30;43 |
N/D |
PS_HDR3_COLOR |
30;45 |
N/D |
PYPI_RUN_PYVER |
(2.7|3.5|3.6|3.7|3.8) |
python versions to run (only PYPI projects) |
SERVER_EXPECTED_ERRORS |
# of expected errors after tests |
|
TEST_DEPENDENCIES |
0 |
Read above |
TRAVIS_DEBUG_MODE |
0 |
Read above |
TRAVIS_PDB |
The value ‘true’ activates pdb in local ‘travis -B’ |
|
UNBUFFER |
1 |
Use unbuffer (colors) to log results |
UNIT_TEST |
Read above |
|
TEST |
Read above |
|
VERSION |
Odoo version to test (see above) |
|
WEBSITE_REPO |
Load package for website tests |
|
WKHTMLTOPDF_VERSION |
0.12.5 |
Version of wkhtmltopdf (value are 0.12.1 |
Debug information
If you declare the following directive in <env global> section:
TRAVIS_DEBUG_MODE=”n”
where “n” means:
Parameter |
0 |
1 |
2 |
3 |
9 |
Informative messages |
|
|
|
|
|
Inspect internal data |
|
|
|
|
|
MQT tests |
|
|
|
|
|
Installation log level |
ERROR |
WARN |
INFO |
INFO |
|
Execution log level |
INFO |
TEST |
TEST |
TEST |
|
Note this feature does not work with OCA MQT. Local test and TravisCI test have slightly different behavior.
When MQT is execute in local environment the value
TRAVIS_DEBUG_MODE=”9”
does not execute unit test. It is used to debug MQT itself.
Tree directory
While travis is running this is the tree directory:
${HOME} # home of virtual environment (by TravisCI)
┣━━ build # build root (by TravisCI)
┃ ┣━━ ${TRAVIS_BUILD_DIR} # testing repository (by TravisCI)
┃ ┗━━ ${ODOO_REPO} # odoo or OCB repository to check with (0) (1) (2)
┃
┣━━ ${ODOO_REPO}-${VERSION} # symlink of ${HOME}/build/{ODOO_REPO} (0) (1)
┃
┣━━ dependencies # Odoo dependencies of repository (0) (3)
┃
┣━━ tools # clone of Zeroincombenze tools (3) (4)
┃ ┃
┃ ┣━━ zerobug # z0bug testing library
┃ ┃ ┗━━ _travis # testing commands
┃ ┗━━ z0bug_odoo # Odoo testing library
┃ ┗━━ travis # testing commands
┃
┗━━ maintainer-quality-tools # OCA testing library
┗━━ travis # testing commands
(0) Same behavior of OCA MQT
(1) Cloned odoo/odoo or OCA/OCB repository to check compatibility of testing modules
(2) If the testing project is OCB, travis_install_env ignore this directory
(3) Done by then following statements in .travis.yml:
- travis_install_env
Above statements replace the OCA statements:
- travis_install_nightly
(4) Done by following statements in .travis.yml::
- git clone https://github.com/zeroincombenze/tools.git ${HOME}/tools --depth=1
- \${HOME}/tools/install_tools.sh -qpt
- source ${HOME}/devel/activate_tools -t
Above statements replace OCA following statements:
- git clone https://github.com/OCA/maintainer-quality-tools.git ${HOME}/maintainer-quality-tools --depth=1
- export PATH=${HOME}/maintainer-quality-tools/travis:${PATH}
TestEnv: the test environment
TestEnv makes available a test environment ready to use in order to test your Odoo module in quick and easy way.
The purpose of this software are:
Create the Odoo test environment with records to use for your test
Make available some useful functions to test your module (in z0bug_odoo)
Simulate the wizard to test wizard functions (wizard simulator)
Environment running different Odoo modules versions
Please, pay attention to test data: TestEnv use internal unicode even for python 2 based Odoo (i.e. 10.0). You should declare unicode date whenever is possible. Note, Odoo core uses unicode even on old Odoo version.
Tests are based on test environment created by module mk_test_env in repository https://github.com/zeroincombenze/zerobug-test
Requirements
Ths TestEnv software requires:
python_plus PYPI package
z0bug_odoo PYPI package
python 2.7 / 3.6 / 3.7 / 3.8
TestEnv is full integrated with Zeroincombenze® tools. See https://zeroincombenze-tools.readthedocs.io/ and https://github.com/zeroincombenze/tools.git Zeroincombenze® tools help you to test Odoo module with pycharm.
This module is part of tools project.
Last Update / Ultimo aggiornamento: 2023-04-15
