OXIDwiki

Certification/Modules

From OXIDwiki

Jump to: navigation, search
PAGE_TITLE: Q2VydGlmaWNhdGlvbiBvZiBPWElEIGVTaG9wIE1vZHVsZXM=

Contents

General

What do I do when I have any question regarding certification?

If you have any questions regarding certification procedure, please write to email address modulecertification@oxid-esales.com.

Why should modules be certified?

It's recommended to certify modules which you intend to offer in OXID eXchange. Certified modules are presented more flashy than non-certified modules, promoted in OXID's newsletters and get special seal which indicates their certification as well as proves tested quality.

What extensions can be certified?

You have to provide automated tests when you request certification. These tests are essential to pass certification. This means, all extensions which have automated tests can be certified.

What knowledge do you need to certify extensions?

You need basic knowledge in

  • writing modules for OXID eShop
  • Writing unit tests for your modules
  • checking out SVN
  • how to use a Linux shell
  • how to copy files from your local PC to a virtual Linux machine
  • how to install software on a Linux box

Setting up VM

Do these steps to get VM up and running for module certification:

  1. Download and install the OXID-VM from OXIDforge and install it.
  2. Increase RAM of the VM to e.g. 1024M
  3. Get eShop and eShop Tests from SVN . Be sure they match and have same revision numbe. For OXID eShop Community Edition you can find these at http://svn.oxid-esales.com/trunk/
  4. Extract them
       e.g.
        eshop: /var/www/oxid_testing
        tests: /var/www/tests
  5. Install eShop with demodata using normal eShop setup (Step 2: choose Germany,Austria...).
    Also, do not choose UTF-8 mode (for running the main test suite) - tests are created for eShop demodata in latin1 encoding.
    UTF-8 functionality is tested in a separate tests suite - unitUtf8 (which requires UTF-8 eShop to be used).
  6. Download PHPUnit 3.4 version from http://pear.phpunit.de/ and extract it to <PHPUNIT PLACE>
    (the one from normal pear installation should also work ok)
    Note: tested PHPUnit version is 3.4.3 (currently used by eShop dev team) and 3.4.9 (tested once)
  7. After installation, copy the setup folder from your eShop package in the root of your installed shop (Setupfiles are required for unittests, but deleted after installation)
  8. Optional, but strongly recommended:
    in eShop/config.inc.php change:
       $this->iDebug = -1;
    This will enable throwing exceptions on wrong SQL queries, so tests will run more precisely and failures shown will be clearer.
  9. Check file permission. (eShop and test files must be writteable)
    e.g.
        chown -R www-data * (eShop and test /)
        chown www-data .htaccess (eShop /)
        chmod -R 0775 * (eShop and test /)
        chmod 0775 .htacces (eShop /)

Basic setup for tests is done now.

Executing tests

Run tests from the tests directory using a user which has write permissions to eShop and tests files.

cd <TEST PLACE>

oxPATH=<ESHOP PLACE> oxADMIN_PASSWD='<ADMIN PW>' \ 
php -d 'memory_limit=1024M' -d    'include_path=.:<PHPUNIT PLACE>' \
<PHPUNIT PLACE>/phpunit.php --verbose --bootstrap bootstrap.php \
<TEST UNIT>

Note: When PHPUnit-3.4.3 version is used, include path should look this way: 

'include_path=.:<PHPUNIT PLACE>/PHPUnit:<PHPUNIT PLACE>'

e.g.

for a single test unit:

cd /var/www/tests

su www-data

oxPATH=/var/www/oxid_testing/ oxADMIN_PASSWD='securesafe' \
php -d 'memory_limit=1024M' -d 'include_path=.:/home/oxid/PHPUnit-3.4.9' \
/home/oxid/PHPUnit-3.4.9/phpunit.php --verbose --bootstrap bootstrap.php \
unit/accountTest.php

for all tests:

cd /var/www/tests

su www-data

oxPATH=/var/www/oxid_testing/ oxADMIN_PASSWD='securesafe' \
php -d 'memory_limit=1024M' -d 'include_path=.:/home/oxid/PHPUnit-3.4.9' \
/home/oxid/PHPUnit-3.4.9/phpunit.php --verbose --bootstrap bootstrap.php \
AllTestsUnit

Generating code coverage report

  • Edit bootstrap.php file in the module tests directory and ensure that the phpunit filter whitelist includes only your module files.

e.g.

if (getenv('CODECOVERAGE')) {
    // PHPUnit_Util_Filter configuration
    PHPUnit_Util_Filter::$addUncoveredFilesFromWhitelist = true;

    PHPUnit_Util_Filter::addDirectoryToWhitelist(oxPATH.'modules/mymodule');
    PHPUnit_Util_Filter::removeFileFromWhitelist(oxPATH.'modules/mymodule/3rdpartyfile.php');
    PHPUnit_Util_Filter::addFileToWhitelist(oxPATH.'someotherplace/othermodulefile.php');
    PHPUnit_Util_Filter::removeDirectoryFromWhitelist(oxPATH.'/modules/mymodule/3rdpartydir/');
}

This ensures only module files will be displayed in the code coverage report, also, not tested files will be included in the report too.

To generate OXID eShop code coverage, leave bootstrap.php with the setup from the tests package.

  • Now you can create code coverage report 
oxPATH=/var/www/oxid_testing/ CODECOVERAGE=1 oxADMIN_PASSWD='securesafe' php -d \
'memory_limit=1024M' -d 'include_path=.:/home/oxid/PHPUnit-3.4.9' \
/home/oxid/PHPUnit-3.4.9/phpunit.php --verbose --bootstrap bootstrap.php \
--coverage-html report_dir AllTestsUnit
  • "report_dir" is the name of the directory where the code coverage report will be generated
  • Set CODECOVERAGE=1 to exclude 3rd Party Modules. (behind oxPATH)
  • After the generation you'll find a lot of *.html files. The entry point to the report is the "index.html". You can call it in browser by this URL: http://youripaddress/reports/

Getting code metrics

First, setup module tests' bootstrap.php file like described in the "Generating code coverage report" section, and check if the code coverage generation works correctly. Now you're able to calculate code metrics: 

oxPATH=/var/www/oxid_testing/ CODECOVERAGE=1 oxADMIN_PASSWD='securesafe' php -d \
'memory_limit=1024M' -d 'include_path=.:/home/oxid/PHPUnit-3.4.9' \
/home/oxid/PHPUnit-3.4.9/phpunit.php --verbose --bootstrap bootstrap.php \
--log-metrics report.xml AllTestsUnit

A file named "report.xml" will be generated that contains the code metrics

Interpreting the code metrics

To interpret the code metrics we offer a script called "MC_Metrics.php which is located in the root directory of Unittest package you've got from SVN. Start it by calling "php MC_Metrics.php METRIC_XML_FILE". As a result you'll get the totalaverage ("AVG") and the average over for each class. The most interesting values are the total averages. These values should be smaller then the values below.

Allowed values for code metrics

  • Code coverage > 90%
  • CRAP Index < 30
  • NPATH Complexity < 200
  • Cyclomatic Complexity < 4

Explanation of the code metrics

  • Code Coverage
    This value tells you how much percent of code lines are covered by tests
  • CRAP - Index
    The Change Risk Analysis and Predictions (CRAP) index of a function or method uses cyclomatic complexity and code coverage from automated tests to help estimate the effort and risk associated with maintaining legacy code. A CRAP index over 30 is a good indicator of crappy code.
  • NPATH Complexity
    The NPATH metric computes the number of possible execution paths through a function. It takes into account the nesting of conditional statements and multi-part boolean expressions (e.g., A && B, C || D, etc.). o A NPATH Complexity over 200 is a good candidate for a closer decomposition
  • Cyclomatic Complexity
    Checks cyclomatic complexity against a specified limit. The complexity is measured by the number of if, while, do, for, ?:, catch, switch, case statements, and operators && and || (plus one) in the body of a constructor, method, static initializer, or instance initializer. It is a measure of the minimum number of possible paths through the source and therefore the number of required tests. Generally 1-4 is considered good, 5-7 ok, 8-10 consider re-factoring, and 11+ re-factor now!

Request for certificate

Now you've got nearly everything you need. Next step is getting a price.

Just fill in determined code metrics into price calculation chart. As soon you're satisfied with the price, email your module, tests, installation instructions to modulecertification@oxid-esales.com and ask for certification.

If you have any questions regarding certification procedure, you can also write to this email address.

Tips & Tricks

Group multiple test files into one run

  • Make a copy of the file "AllTests.php" and name it something like "MyProjectAllTests.php"
  • In line 65 you then adapt the pattern to match only your test files.
  • E.g. if your test files are all named "fooSOMETHINGTest.php" you can use a filter like "glob("$sDir/foo*Test.php") as …" 
  • ./phpunit MyProjectAllTest will now run all your test files at once.
  • To run all the unittests you obtained from the partner svn type: ./phpunit AllTests


FAQ

Question: Test fail and tell there are classes fb... missing. What's wrong?

Answer: Unfortunately there is currently no cURL installed on that VM. Please install it. Next release of VM will come prepared with cURL


Question: If i try to run all tests, they fail due to low memory. What's wrong?

Answer: To run all OXID eShop standard tests you need 2 GB of RAM. VM comes with 512 MB. Increase available RAM.


Question: Standard OXID eShop tests fail and I cannot recognize any reason. What can I do?

Answer: Make sure you ran all tests on a fresh database. Any changes to database (even previously failed tests) can harm proper test passing.


Question: Do I have to use this VM?

Answer: No. You don't. It's not important where tests or mc_metrics.php run. It's only important that you have all the results from mc_metrics.php.


Question: I have several modules which have basically the same functionality, just differ in a very slight way. Do I have to pass certification for all of them separately?

Answer: Depends :). If it's really a very small change and there are tests for it: Most likely you do not have to pass certification for all of them separately. In doubt: just, see above.


Question: I have some extension for OXID eShop which is just some CSS and/or HTML. How can I pass certification?

Answer: You cannot. You need unit tests and PHP code to pass certification. If one of these is missing, certification is not possible.

Retrieved from "http://wiki.oxidforge.org/Certification/Modules"
Personal tools