Testing framework for Black Fin on u Clinux

Зарегистрируйтесь, чтобы просмотреть полный документ!

Testing framework for Black. Fin on u. Clinux Grace Pan

Testing on Blackfin Toolchain Use testing: Dejagnu to ease/automate 60, 000+ Toolchain tests on simulator, hardware, and hardware with fdpic format. All features of the toolchain (C, C++, gdb, gas, binutils, etc. ) are ‘exercised’ Kernel All testing tests run on different targets: BF 533 -STAMP, BF 537 -STAMP, BF 561 EZKIT, BF 533 -EZKIT, even custom boards. Run Linux Test Project (LTP)(about 600 cases), which is a collection of test suites to validate the reliability, robustness, and stability of Linux. Blackfin/u. Clinux is the first architecture to run LTP on u. Clinux/u. Clibc/nommu architecture. Automated stress tests on kernel and device drivers using expect scripts. Boot. Loader Tested on BF 533 -STAMP and BF 537 -STAMP of uboot commands and features. All tests are Well documented on docs. blackfin. uclinux. org and can be reproduced

Uclinux-dist testsuites under uclinux-dist source tree testsuites: Run a certain case, rtc, for example: kernel_test : a line that connects all the cases under this folder to run one by one. kernel_config. exp: the file that sets the common used params. xxx/build_xxx_kernel. exp: to build a kernel for a certain xxx case. xxx/xxx_test. exp : auto script for xxx testing. test_scripts/run_kernel_test: run kernel_test and call summary and compare scripts. test_scripts/get_test_summary: do summary for each round of testing. test_scripts/compare_kernel_results: compare test results of two rounds of testing. test_scripts/test_runtime_control: kill the case that runs abnormal and lasts too long. cd testsuites/rtc. Run . /build_rtc_kernel. exp BF 537 -STAMP Run. /rtc_test. exp. Log files are generated as build_rtc_kernel_log and rtc_test_log. Run the testcases on regression machine: cd /home/test_scripts/uclinux-dist cp run_kernel_test to this folder and set the particular value for the testing params. Since this file is different for each testing bench, so it is not updated and copied out when setting up the testing, no much modification. do. /run_kernel_test.

Toolchain Testing machine: 10. 99. 22. 40 - toolchain 3. 4 BF 537, host Suse 10. 99. 22. 41 - toolchain 4. 1 BF 537, host Suse 10. 99. 22. 42 - toolchain 4. 1 BF 533, host Suse 10. 99. 22. 43 - toolchain 4. 1 BF 561, host Suse 10. 99. 22. 146 - toolchain 4. 1 BF 537, host Mac Uboot Testing machine: 10. 99. 22. 58 - BF 533, host Suse 10. 99. 22. 59 - BF 537, host Suse

Uclinux-dist Testing machine: 10. 99. 22. 50 - BF 537 -STAMP-CF, FLAT 10. 99. 22. 51 - BF 537 -STAMP-STD, FLAT 10. 99. 22. 52 - BF 537 -STAMP-AD 73311, ELF 10. 99. 22. 54 - BF 537 -STAMP-ADV 7393, FLAT 10. 99. 22. 55 - BF 533 -STAMP-STD, FLAT 10. 99. 22. 56 - BF 533 -STAMP-STD, ELF 10. 99. 22. 57 - BF 537 -DIGIUM-PBX, FLAT 10. 99. 22. 46 - BF 533 -EZKIT-STD, FLAT 10. 99. 22. 47 - BF 561 -EZKIT-STD, FLAT

Cruise. Control Overview Cruise. Control is a framework for a continuous build process. It includes, but is not limited to, plugins for email notification, Ant, and SVN, etc. Cruise. Control 2 main modules: the build loop: core of the system, it triggers build cycles then notifies various listeners (users) using various publishing techniques. The trigger can be internal (scheduled or upon changes in a SCM) or external. It is configured in a xml file which maps the build cycles to certain tasks, thanks to a system of plugins. Depending on configuration, it may produce build artifacts. the legacy reporting allows the users to browse the results of the builds and access the artifacts Cruisecontrol finds bugs during development process, not at the last stage of testing. Main page: http: //cruisecontrol. sourceforge. net/index. html

Set up Cruise. Control Download cruisecontrol-xxx-2. x. zip from the website. cd /home/test/work, unzip the zip package. Follow the instructions in page: http: //cruisecontrol. sourceforge. net/gettingstarted. html to build and install cruisecontrol. cd /home/test/work/cruise/checkouts/. Check out the project svn source tree here, for example uclinux-dist project. cd /home/test/work/cruise/, edit config. xml, build-uclinux-dist. xml, checkouts/uclinuxdist/build. xml to fit into this testing. mkdir /home/test/work/cruise/test_scripts/uclinux-dist/, copy run_kernel_test over as the startup script from uclinux-dist source tree subfolder testsuites/test_scripts. cd /home/test/work/cruise/, run. /main/bin/cruisecontrol. sh. It would use the config. xml in that folder to read the config params. The log files would be saved at test_scripts/uclinux-dist/logs and part of it would be copied to logs/uclinux-dist/. Add gforge jars if the result need to be reported to gfore website.

Cruisecontrol config. xml <cruisecontrol> <project name="uclinux-dist" buildafterfailed="true"> <plugin name="gforge" classname="org. gforge. cruisecontrol. publishers. GForge. SOAPExternal. Publisher" host="blackfin. uclinux. org/" ssl="false" /> <bootstrappers> <svnbootstrapper local. Working. Copy="checkouts/${project. name}" /> … </bootstrappers> <schedule interval="60"> <ant anthome="/home/test/work/cruise/apache-ant-1. 6. 0" buildfile="build-uclinux-dist. xml" target="build" uselogger="true" usedebug="false" /> </schedule> <log dir="logs/${project. name}"/> <publishers> … </publishers> </project> </cruisecontrol>

build-uclinux-dist. xml <project name="build-uclinux-dist" default="build" basedir="checkouts/uclinux-dist"> <target name="build"> <ant antfile="build. xml" target="build"/> </target> </project> build. xml  <project name="uclinux-dist" default="build"> <target name="build" > <exec dir="" executable="/bin/sh" os="Linux, Solaris, Mac OS X" > <arg value=". . /test_scripts/uclinux-dist/run_kernel_test"/> </exec> </target> </project>

Execution sequence: . /main/bin/cruisecontrol. sh ↓ config. xml ↓ build-uclinux-dist. xml ↓ checkouts/uclinux-dist/build. xml ↓ run_kernel_test ↓ kernel_test

Cruisecontrol starting up. /main/bin/cruisecontrol. sh Using Cruise Control at /home/test/work/cruise/main /usr/lib/jvm/java/bin/java -cp /usr/lib/jvm/java/lib/tools. jar: /home/test/work/cruise/main/dist/cruisecontrol. jar: /home/test/work/cruise/main/lib/log 4 j. jar: /ho me/test/work/cruise/main/lib/jdom. jar: /home/test/work/cruise/main/lib/ant/antlauncher. jar: /home/test/work/cruise/main/lib/xerces. Impl-2. 8. 0. jar: /home/test/work/cruise/main/lib/xml-apis 2. 8. 0. jar: /home/test/work/cruise/main/lib/xmlrpc-2. 0. 1. jar: /home/test/work/cruise/main/lib/xalan 2. 7. 0. jar: /home/test/work/cruise/main/lib/serializer-2. 7. 0. jar: /home/test/work/cruise/main/lib/jakarta-oro 2. 0. 3. jar: /home/test/work/cruise/main/lib/mail. jar: /home/test/work/cruise/main/lib/junit. jar: /home/test/work/cruise/main/lib/a ctivation. jar: /home/test/work/cruise/main/lib/commons-net-1. 1. 0. jar: /home/test/work/cruise/main/lib/starteamsdk. jar: /home/test/work/cruise/main/lib/mx 4 jtools. jar: /home/test/work/cruise/main/lib/mx 4 jremote. jar: /home/test/work/cruise/main/lib/smack. jar: /home/test/work/cruise/main/lib/comm. jar: /home/test/work/cruise/main /lib/commons-logging. jar: /home/test/work/cruise/main/lib/x 10. jar: /home/test/work/cruise/main/lib/fastmd 5. jar: /home/test/work/cruise/main/lib/maven-embedder-2. 0. 3 dep. jar: /home/test/work/cruise/main/lib/axis. jar: /home/test/work/cruise/main/lib/jaxrpc. jar: /home/test/work/cruise/main/lib/ wsdl 4 j-1. 5. 1. jar: /home/test/work/cruise/main/lib/commons-codec-1. 3. jar: /home/test/work/cruise/main/lib/commonsdiscovery-0. 2. jar: /home/test/work/cruise/main/lib/commons-httpclient-3. 0 rc 3. jar: /home/test/work/cruise/main/lib/crimson. jar: /home/test/work/cruise/main/lib/saaj. jar: /home/test/work/cruise/main/lib/ gforge-external-publisher. jar: . -Djavax. management. builder. initial=mx 4 j. server. MX 4 JMBean. Server. Builder Cruise. Control [cc]Jun-21 17: 48: 59 Main - Cruise. Control Version 2. 5 Compiled on June 1 2007 1205 [cc]Jun-21 17: 48: 59 LConfig. Manager- reading settings from config file [/home/test/work/cruise/config. xml] [cc]Jun-21 17: 48: 59 trol. Controller- project. Name = [uclinux-dist] [cc]Jun-21 17: 48: 59 trol. Controller- No previously serialized project found [/home/test/work/cruise/uclinux-dist. ser], forcing a build. [cc]Jun-21 17: 48: 59 LConfig. Manager- using settings from config file [/home/test/work/cruise/config. xml] [cc]Jun-21 17: 48: 59 Project - Project uclinux-dist starting [cc]Jun-21 17: 48: 59 Project - Project uclinux-dist: idle [cc]Jun-21 17: 48: 59 Project - Project uclinux-dist started [cc]Jun-21 17: 48: 59 Project - Project uclinux-dist: in build queue [cc]Jun-21 17: 48: 59 Build. Queue - Build. Queue started

Log 20070615145327. xml: <? xml version="1. 0" encoding="UTF-8"? > <cruisecontrol> <modifications> <modification type="svn"> <file action="modified"> <revision>5394</revision> <filename>/trunk/lib/ffmpeg-svn-8891/libavutil/common. h</filename> </file> <date>06/12/2007 19: 26: 33</date> <user>mmh</user> <comment><![CDATA[[#3265] adding blackfin specific read_time profile support]]></comment> <revision>5394</revision> </modification> …. </modifications> <info> <property name="projectname" value="uclinux-dist" /> <property name="lastbuild" value="20070612175012" /> …… <property name="logdir" value="/home/test/work/cruise/logs/uclinux-dist" /> <property name="logfile" value="log 20070615145327. xml" /> </info> <build time="20 minutes 1 second"> <target name="build" time="20 minutes 1 second"> <task location="/home/test/work/cruise/build-uclinux-dist. xml: 5: " name="ant" time="20 minutes 1 second"> <target name="build" time="20 minutes 0 seconds"> <task location="/home/test/work/cruise/checkouts/uclinux-dist/build. xml: 5: " name="exec" time="20 minutes 0 seconds"> <message priority="info"><![CDATA[Here is the difference between lastrun and thisrun test results: ]]></message> <message priority="info" /> …… <message priority="info"><![CDATA[Building kernel that now fail, but passed before: ]]></message> </task> </target> </build> </cruisecontrol>

Changes in Cruise. Control Add test result judgement in cruise/main/src/net/sourceforge/cruisecontrol/util/XMLLog. Helper. java: public boolean is. Build. Successful() { String filename = "/home/test/work/cruise/test_scripts/uclinux-dist/logs/thisrun/test_summary/test_results"; File. Reader fr = null; try { fr = new File. Reader(filename); Buffered. Reader br = new Buffered. Reader(fr); String line = null; String result = null; line = br. read. Line(); while (line != null) { result = line. trim(); if (result. equals("success")) { return true; } else if (result. equals("failed")) { return false; } } br. close(); fr. close(); } catch (IOException e) { e. print. Stack. Trace(); } return (log. get. Child("build"). get. Attribute("error") == null); }

Add detailed log in html mail. The xsl scripts to translate the xml log into html is under cruise/reporting/jsp/webcontent/xsl. Add in buildresults. xsl : <xsl: import href=“build. xsl”/> … <p><xsl: apply-templates select=“$cruisecontrol. list” mode=“build” /></p> Create build. xsl according to the template of errors. xsl.

Example of cruisecontrol log mail [Uclinux-dist-test-notify] uclinux-dist Test Failed BUILD COMPLETE - build. 3 Date of build: 06/19/2007 12: 07: 28 Time to build: 1, 560 minutes 45 seconds Last changed: 06/19/2007 09: 10: 31 Last log entry: [#2443] Update test scripts. Log: Testing on 10. 99. 22. 50 with BF 537 -STAMP-CF (BF 537) flat slab(p 2) ICACHE(on) DCACHE(on) write_through dma Wed Jun 20 05: 07: 57 CST 2007. Process: ltp_auto_test. e, ID: 21457 to be killed. Here is the difference between lastrun and thisrun test results: Building kernel that now fail, but passed before: build_pbx_kernel_log build_pound_kernel_log #####Test Summary #######The following build kernel fails: ####### build_linphone_kernel_log ###### Pass Num is: 27. ####### DLL TEST DPMC TEST … Modifications since last successful build: (70) modifiedvapier/trunk/include/asm-blackfin/unistd. h 06/18/2007 23: 04 move cond_syscall() behind __KERNEL__ like all other architectures modifiedvapier/trunk/scripts/checkstack. pl 06/18/2007 23: 18: 45 fill in the Blackfin specific pieces in the check stack script modifiedadamliyi/trunk/user/blkfin-apps/linphone-1. 6. 0/console/linphonec. c 06/16/2007 06: 33: 43 for task [#2386], add video display in linphone …

Web page Publisher If you want to see it from the web pages, like : http: //10. 99. 22. 164: 8080/cruisecontrol/buildresults/uclinuxdist? log=log 20070215173903 Lbuild. 1 (This link is configured in config. xml) , you need to set the write permission of cruise/logs/uclinux-dist, since it dynamically convert xml to html and put them under this folder name _cache by default. Also you need set up tomcat to hold these jsp pages.

Cruise. Control Merits: Allows you: you to see what is happening in the source tree. It shows who checked in what (by showing modifications); Test of what config has built and test successfully; what platforms are broken and how they are broken (the build logs); With this - the system can automatically determine who broke the build, so fixes can occur quickly

Thank You. 19

Скачать презентацию Testing framework for Black Fin on u Clinux

a4d6c43c40e4c7aa8d5fe97ddd628407.ppt

Количество слайдов: 19