cookutils annotate doc/cookutils.en.html @ rev 525

Add cookiso.conf.
author Christopher Rogers <slaxemulator@gmail.com>
date Fri Aug 24 22:05:20 2012 +0000 (2012-08-24)
parents b5f2e3c5d984
children
rev   line source
pankso@19 1 <!DOCTYPE html>
pankso@19 2 <html xmlns="http://www.w3.org/1999/xhtml">
pankso@19 3 <head>
pankso@19 4 <title>Cookutils Documentation</title>
pankso@19 5 <meta charset="utf-8" />
pankso@19 6 <link rel="stylesheet" type="text/css" href="style.css" />
pankso@19 7 </head>
pankso@19 8 <body>
pankso@19 9
pankso@19 10 <div id="header">
pankso@19 11 <h1>Cookutils Documentation</h1>
pankso@19 12 </div>
pankso@19 13
pankso@19 14 <!-- Start content -->
pankso@19 15 <div id="content">
pankso@19 16
pankso@19 17 <h2>SliTaz Cook &amp; Cooker</h2>
pankso@19 18
pankso@19 19 <p>
paul@53 20 The SliTaz Cookutils provide tools and utils to help build SliTaz packages. They
pankso@19 21 are easy to use and learn, fast and light. You will be able to create SliTaz
paul@53 22 packages in a few commands. The cookutils provide the 'cook' utility and the
pankso@19 23 <a href="#cooker">Cooker</a>.
pankso@19 24 </p>
pankso@19 25 <p>
paul@53 26 Cook lets you compile and create a package, provide a log file and check the
pankso@19 27 receipt/package quality. The Cooker is a build bot with more automation
paul@179 28 and can be used as a frontend to cook since it provides a CGI/web interface
paul@53 29 which lets you view cook logs in a nice and colored way. Cook and the Cooker
paul@179 30 use the same DB files and wok, they both share <a href="#blocked">blocked</a>
paul@53 31 and broken packages as well as any activity.
pankso@19 32 </p>
pankso@261 33 <p>
paul@263 34 For technical information, for example the coding style, etc, please refer to the
paul@263 35 README found in the source tree or in /usr/share/doc/cookutils.
pankso@261 36 </p>
pankso@19 37
pankso@27 38 <h3>Cook usage</h3>
pankso@27 39 <p>
paul@53 40 Cook provides a small built-in help usage that you can display with the
paul@53 41 command 'usage'. It also has some options to perform special tasks on
paul@174 42 a package before cooking it or afterwards. To get help and usage:
pankso@27 43 </p>
pankso@27 44 <pre>
pankso@27 45 # cook usage
pankso@27 46 </pre>
pankso@27 47
paul@53 48 <h3>Howto</h3>
pankso@19 49 <p>
paul@53 50 The first thing you will have to do before building packages is setup
paul@53 51 your environment. The 2 recommended ways of working: cook directly on host
pankso@19 52 or cook in chroot to protect your host. In the case you want to work in a
paul@53 53 chroot you can install and use Tazdev to create one and chroot into it:
pankso@19 54 </p>
pankso@19 55 <pre>
pankso@19 56 # tazdev gen-chroot &amp;&amp; tazdev chroot
pankso@19 57 </pre>
pankso@27 58 <p>
paul@53 59 By default Tazdev creates a chroot in /home/slitaz/cooking/chroot but you
paul@53 60 can specify a custom path in the argument. The chroot location is not
pankso@40 61 important, when you will be in the chroot you will use standard SliTaz
paul@53 62 paths such as /home/slitaz/wok for the wok directory or /home/slitaz/log
paul@174 63 for all the cook logs. As usual you can display tazdev help usage with:
pankso@40 64 tazdev usage.
pankso@40 65 </p>
pankso@40 66 <p>
paul@53 67 When you use a chroot there are 2 special directories mounted with the bind
pankso@40 68 option: src and packages. The sources for all packages are stored by default
pankso@40 69 in /home/slitaz/src, this directory is mounted into the chroot so the utils
paul@53 70 can use them. This method lets you share sources between many chroots such
pankso@40 71 as one for cooking and one for stable. The packages directory default
pankso@40 72 location is: /home/slitaz/[version]/packages so they are not in the chroot
paul@53 73 and are safe in case the chroot is removed by error.
pankso@27 74 </p>
pankso@19 75
pankso@19 76 <h3>Getting started</h3>
pankso@19 77 <p>
paul@53 78 So you have decided the way you want to work, so lets prepare the cook environment.
paul@53 79 Cook uses the cook.conf configuration file, if you want to use custom paths for
paul@53 80 SliTaz directories and files, you'll have to modify it. The setup will create
paul@53 81 some directories and files to keep trace of activity and errors, all files
pankso@19 82 are pure plain text files that you can open in a text editor. To prepare
paul@53 83 your environment:
pankso@19 84 </p>
pankso@19 85 <pre>
pankso@19 86 # cook setup
pankso@19 87 </pre>
pankso@27 88 <p>
paul@53 89 The setup command has a --wok option which lets you clone a SliTaz wok while
paul@179 90 setting up your cook environment. Even if you are not yet an official developer
paul@53 91 you can clone it and use existing packages as an example to create your own.
pankso@236 92 To setup and clone the default cooking wok or the undigest wok:
pankso@27 93 </p>
pankso@27 94 <pre>
pankso@27 95 # cook setup --wok
pankso@236 96 # cook setup --undigest
pankso@27 97 </pre>
pankso@19 98
pankso@19 99 <h3>Test your environment</h3>
pankso@19 100 <p>
paul@53 101 Cook provides a test command which will create a package and cook it. This lets
paul@174 102 you see if your environment is working and it provides an example package with
paul@53 103 a receipt. The dummy package is named 'cooktest' and can be removed after
paul@174 104 testing. To cook the test package:
pankso@19 105 </p>
pankso@19 106 <pre>
pankso@19 107 # cook test
pankso@19 108 </pre>
pankso@19 109
pankso@19 110 <h3>Create and cook</h3>
pankso@19 111 <p>
paul@174 112 If your environment is setup correctly you can start creating and compiling
paul@197 113 SliTaz packages from your wok. To create a new package with an empty receipt
paul@197 114 (you can also create a receipt interactively):
pankso@19 115 </p>
pankso@19 116 <pre>
pankso@19 117 # cook new pkgname
paul@197 118 # cook new pkgname --interactive
pankso@19 119 </pre>
pankso@19 120 <p>
paul@53 121 If you have just created a new package, you'll have to edit the receipt with your
paul@174 122 favorite text editor. When the receipt is ready or if you have an existing
paul@174 123 package, you can cook it:
pankso@19 124 </p>
pankso@19 125 <pre>
pankso@19 126 # cook pkgname
pankso@19 127 </pre>
pankso@19 128 <p>
paul@174 129 If all went well you will find your package in the $SLITAZ/packages
paul@53 130 directory and any produced files in $SLITAZ/wok/pkgname.
pankso@39 131 </p>
pankso@39 132
pankso@39 133 <h3>Cook and install</h3>
pankso@39 134 <p>
pankso@39 135 If you want to cook and install the package in one command:
pankso@19 136 </p>
pankso@19 137 <pre>
pankso@19 138 # cook pkgname --install
pankso@19 139 </pre>
pankso@19 140
pankso@39 141 <h3>Get sources</h3>
pankso@39 142 <p>
paul@53 143 If you want or need to download only the source of a package without
paul@53 144 building it, you can use the option --getsrc as below:
pankso@39 145 </p>
pankso@39 146 <pre>
pankso@39 147 # cook pkgname --getsrc
pankso@39 148 </pre>
pankso@39 149
pankso@19 150 <h3>Clean packages</h3>
pankso@19 151 <p>
paul@53 152 After compilation and packaging there are several files in the wok that take up
pankso@19 153 disk space. To clean a single package:
pankso@19 154 </p>
pankso@19 155 <pre>
pankso@19 156 # cook pkgname --clean
pankso@19 157 </pre>
pankso@19 158 <p>
pankso@19 159 You can also clean the full wok at once or you can choose to keep SliTaz
pankso@19 160 related files and just remove the source:
pankso@19 161 </p>
pankso@19 162 <pre>
pankso@19 163 # cook clean-wok
pankso@19 164 # cook clean-src
pankso@19 165 </pre>
pankso@19 166
pankso@69 167 <h3>Search</h3>
pankso@69 168 <p>
paul@89 169 Cook provides a simple search function to quickly find a package in the
paul@89 170 wok. It uses grep and so supports regular expressions:
pankso@69 171 </p>
pankso@69 172 <pre>
pankso@69 173 # cook search busybox
pankso@69 174 </pre>
pankso@69 175
slaxemulator@482 176 <h3>Unbuild</h3>
slaxemulator@482 177 <p>
slaxemulator@482 178 Unbuild is to provide a list of packages you don't have build based on
slaxemulator@482 179 fullco.txt file. It will echo a list of packages that you have not build:
slaxemulator@482 180 (Note: This also makes a unbuild file in your cache folder.)
slaxemulator@482 181 </p>
slaxemulator@482 182 <pre>
slaxemulator@482 183 # cook unbuild
slaxemulator@482 184 </pre>
slaxemulator@482 185 <p>
slaxemulator@482 186 If you want a more full list with wanted packages use this option:
slaxemulator@482 187 </p>
slaxemulator@482 188 <pre>
slaxemulator@482 189 # cook unbuild --full
slaxemulator@482 190 </pre>
slaxemulator@482 191
pankso@236 192 <h3>Packages DB list</h3>
pankso@19 193 <p>
paul@53 194 Cook can list packages in the wok and also create a suitable packages list
paul@175 195 for Tazpkg. This lets you create a local packages repository quite easily
paul@53 196 and is used to create the official SliTaz packages list found on the mirrors.
paul@53 197 To list the current wok used by cook (you don't need to be root):
pankso@19 198 </p>
pankso@19 199 <pre>
pankso@19 200 $ cook list-wok
pankso@19 201 </pre>
pankso@19 202 <p>
paul@243 203 When creating the packages DB, cook will check if you have a flavors repo in
paul@243 204 /home/slitaz/flavors, if so, it will pack all flavors using the latest
pankso@236 205 packages list available. To create a packages list and the Live flavors
pankso@236 206 files:
pankso@19 207 </p>
pankso@19 208 <pre>
pankso@236 209 # cook pkgdb
pankso@19 210 </pre>
slaxemulator@482 211 <p>
slaxemulator@482 212 To update DB for incoming or packages do this:
slaxemulator@482 213 </p>
slaxemulator@482 214 <pre>
slaxemulator@482 215 # cook pkgdb incoming
slaxemulator@482 216 # cook pkgdb packages
slaxemulator@482 217 </pre>
slaxemulator@482 218
slaxemulator@482 219 <p>
slaxemulator@482 220 For the most part pkgdb rebuilds wanted.txt, depends.txt, and libraries.txt.
slaxemulator@482 221 But to build or update fullco.txt in packages you need to do this:
slaxemulator@482 222 </p>
slaxemulator@482 223 <p>
slaxemulator@482 224 NOTE: fullco.txt is the full cookorder to help rebuild slitaz from scratch
slaxemulator@482 225 and slitaz source dvd.
slaxemulator@482 226 </p>
slaxemulator@482 227 <pre>
slaxemulator@482 228 # cook gen-wok-db
slaxemulator@482 229 </pre>
pankso@19 230
pankso@19 231 <a name="cooker"></a>
pankso@19 232 <h3>The Cooker</h3>
pankso@19 233 <p>
paul@53 234 The Cooker is a Build Bot, its first function is to check for commits in a wok,
pankso@19 235 create an ordered cooklist and cook all modified packages. It can also be
paul@53 236 used as a frontend to cook since they both use the same files. The Cooker can
paul@53 237 also be used to cook a big list of packages at once such as all the packages
paul@53 238 in a flavor. The Cooker provides a nice CGI/Web interface that works by
paul@53 239 default on any SliTaz system since it provides CGI support via the Busybox httpd
pankso@19 240 web server.
pankso@19 241 </p>
pankso@27 242 <p>
paul@53 243 The Cooker provides a small built-in help usage and short command switch.
pankso@31 244 For example to display usage you can use:
pankso@27 245 </p>
pankso@27 246 <pre>
pankso@31 247 # cooker usage
pankso@31 248 # cooker -u
pankso@27 249 </pre>
pankso@19 250
pankso@19 251 <h3>Cooker setup</h3>
pankso@19 252 <p>
paul@53 253 Like cook, the Cooker needs a working environment before starting to use it.
paul@53 254 The main difference with the cook environment is that the Cooker needs 2 woks.
paul@53 255 One Hg and clean wok as a reference and one build wok. In this way it is easy
paul@53 256 to compare both woks and get modifications. If you already have a cook
paul@53 257 environment, you must move your wok before setting up the Cooker or it
pankso@31 258 will complain. Setup will also install a set of development packages that
pankso@31 259 can be configured in the cook.conf configuration file and the variable
pankso@31 260 SETUP_PKGS. To setup your cooker environment:
pankso@19 261 </p>
pankso@19 262 <pre>
pankso@31 263 # cooker setup
pankso@19 264 </pre>
pankso@19 265 <p>
paul@174 266 If all went well you now have 2 woks, base development packages installed
paul@53 267 and all needed files created. The default behavior is to check for commits,
pankso@19 268 you can run a test:
pankso@19 269 </p>
pankso@19 270 <pre>
pankso@19 271 # cooker
pankso@19 272 </pre>
pankso@19 273
pankso@19 274 <h3>Cooker cook</h3>
pankso@19 275 <p>
paul@53 276 Again, 2 ways to work now: make changes in the clean Hg wok and launch the
paul@53 277 cooker without any arguments or cook packages manually. The cooker lets you
paul@53 278 cook a single package or all packages of a category or a flavor. You can also
pankso@19 279 try to build all unbuilt packages, but be aware the Cooker was not designed
paul@174 280 to handle thousands of packages.
pankso@19 281 </p>
pankso@19 282 <p>
paul@53 283 To cook a single package which is the same as 'cook pkgname' but with more
pankso@19 284 logs:
pankso@19 285 </p>
pankso@19 286 <pre>
pankso@31 287 # cooker pkg pkgname
pankso@19 288 </pre>
pankso@19 289 <p>
pankso@19 290 To cook more than one package at once you have different kind of choices.
paul@53 291 You can use an existing package such as used for Live flavors, you can also
paul@53 292 use a custom list using the package names listed line by line. Finally you can
paul@53 293 build all packages of a category.
pankso@19 294 </p>
pankso@19 295 <pre>
pankso@31 296 # cooker flavor [name]
pankso@31 297 # cooker list [/path/to/cooklist]
pankso@31 298 # cooker cat [category]
pankso@19 299 </pre>
pankso@79 300 <p>
paul@174 301 The Cooker lets you also recook a specific Hg revision. It's useful in
paul@89 302 production so that if the Build Bot was interrupted while cooking commits, you
pankso@79 303 can then cook packages manually:
pankso@79 304 </p>
pankso@79 305 <pre>
pankso@79 306 # cooker rev 9496
pankso@79 307 </pre>
pankso@19 308
pankso@49 309 <a name="blocked"></a>
pankso@49 310 <h3>Blocked packages</h3>
pankso@49 311 <p>
paul@53 312 Cook and the Cooker handle a file with a list of blocked package so they will
paul@53 313 not cook when commits happen or if a cooklist is used. This is very useful
pankso@49 314 for a Cooker Build Bot in production. When you block or unblock a package
pankso@49 315 you can add a note to the cooknotes. Blocking packages example:
pankso@49 316 </p>
pankso@49 317 <pre>
pankso@49 318 # cook pkgname --block
pankso@49 319 # cooker block pkgname
pankso@49 320 # cooker -n "Blocked pkgname note"
pankso@49 321 </pre>
pankso@49 322 <p>
pankso@49 323 The list of blocked packages are also displayed on the Cooker web interface.
pankso@49 324 To unblock a package you have to use the unblock command or cook --unblock
pankso@49 325 option:
pankso@49 326 </p>
pankso@49 327 <pre>
pankso@49 328 # cook pkgname --unblock
pankso@49 329 # cooker unblock pkgname
pankso@49 330 </pre>
pankso@49 331
pankso@19 332 <h3>Cooker CGI/Web</h3>
pankso@19 333 <p>
paul@53 334 To let you view log files in a nice way, keep trace of activity and help find
pankso@27 335 errors, you can use the Cooker Web interface located by default in the folder
pankso@261 336 /var/www/cooker. If you don't use a chroot and the Busybox httpd web server
pankso@261 337 is running, the web interface will work without configuration and should be
pankso@261 338 reachable at: <a href="http://localhost/cooker/cooker.cgi">
pankso@261 339 http://localhost/cooker/cooker.cgi</a>
pankso@19 340 </p>
pankso@19 341 <p>
pankso@19 342 If you used a chroot environment, you should also install cookutils on your
pankso@19 343 host and modify the SLITAZ path variable. A standard working way is to have
pankso@19 344 a chroot in:
pankso@19 345 </p>
pankso@19 346 <pre>
pankso@19 347 /home/slitaz/cooking/chroot
pankso@19 348 </pre>
pankso@19 349 <p>
paul@53 350 With /etc/slitaz/cook.conf modified as below:
pankso@19 351 </p>
pankso@19 352 <pre>
pankso@19 353 SLITAZ="/home/slitaz/cooking/chroot/home/slitaz"
pankso@19 354 </pre>
pankso@19 355 <p>
pankso@19 356 Note: It's not obligatory to install the cookutils on your host to use the
pankso@261 357 web interface. If you use Lighttpd you can also copy the cooker.cgi and
pankso@261 358 style.css files for example into your ~/Public directory and use a custom
pankso@261 359 cook.conf with it. The advantage of installing cookutils on the host is to
pankso@261 360 get regular updates via the Tazpkg packages manager. Say you have cloned or
pankso@261 361 downloaded the cookutils:
pankso@19 362 </p>
pankso@19 363 <pre>
pankso@27 364 $ cp -a cookutils/web ~/Public/cgi-bin/cooker
pankso@27 365 $ cp -f cookutils/cook.conf ~/Public/cgi-bin/cooker
pankso@19 366 </pre>
pankso@19 367 <p>
paul@53 368 Edit the configuration file: ~/Public/cgi-bin/cooker/cook.conf to set your
paul@53 369 SLITAZ path and you're all done!
pankso@19 370 </p>
pankso@19 371
pankso@24 372 <h3>Cooknotes</h3>
pankso@24 373 <p>
paul@53 374 The cooknotes feature lets you write small personal notes about packaging
paul@53 375 and is useful for collaboration. The cooknotes was coded to let the SliTaz
paul@53 376 Cooker bot maintainers share notes between themselves and other contributors.
paul@53 377 The Cooker can block a package's build or recook packages manually, for example
paul@53 378 it's nice to make a note if a package is blocked so that the maintainer knows why
pankso@24 379 admin did that. Cooknotes are displayed on the web interface and can be
paul@53 380 checked from a cmdline:
pankso@24 381 </p>
pankso@24 382 <pre>
pankso@31 383 # cooker note "Blocked pkgname due to heavy CPU load"
pankso@31 384 # cooker notes
pankso@24 385 </pre>
pankso@19 386
pankso@59 387 <h3>Cooker as a Build Bot</h3>
pankso@59 388 <p>
paul@62 389 The Cooker is designed to be a Built Bot for SliTaz, this means it monitors
paul@62 390 2 woks, updates the Hg wok, gets the differences and cooks all packages that
pankso@84 391 have been committed. The safer and cleaner way to run the Cooker as a Build
pankso@84 392 Bot with cron is to use a chroot environment, but it can run directly on the
pankso@84 393 host if you want.
pankso@84 394 </p>
pankso@84 395 <p>
pankso@84 396 To run The Cooker automatically you must use cron from the chroot and add a
paul@62 397 single line to root crontabs in /var/spool/cron/crontabs. Say you would like
paul@62 398 to run the Cooker every 2 hours:
pankso@59 399 </p>
pankso@59 400 <pre>
pankso@163 401 * */2 * * * /usr/bin/cooker
pankso@163 402 </pre>
pankso@163 403
pankso@163 404 <h3>Cooker BB started at boot</h3>
pankso@163 405 <p>
paul@168 406 The Cooker environment and cron task can automatically be started at boot time.
paul@168 407 You must have the cookutils-daemon installed on the host and use a standard SliTaz
paul@174 408 installation to make it work properly (cooking goes in /home/slitaz/cooking). The
paul@174 409 daemon script will mount any virtual filesystems if needed as well as source and
paul@174 410 packages. Source files are in /home/slitaz/src and bound into the chroot
paul@174 411 so you can share package's sources between several versions (stable, cooking,
paul@168 412 undigest). If the package is not yet installed:
pankso@166 413 </p>
pankso@166 414 <pre>
pankso@166 415 # tazpkg get-install cookutils-daemon
pankso@166 416 </pre>
pankso@166 417 <p>
pankso@166 418 To start the daemon you must have a cron file definition for
paul@168 419 root in the chroot, the daemon script works like all other system daemons
pankso@166 420 and can be handled with:
pankso@163 421 </p>
pankso@163 422 <pre>
pankso@163 423 # /etc/init.d/cooker [start|stop|restart]
pankso@59 424 </pre>
pankso@59 425
pankso@19 426 <!-- End content -->
pankso@19 427 </div>
pankso@19 428
pankso@19 429 <div id="footer">
pankso@19 430 Copyright &copy; 2011 SliTaz contributors
pankso@19 431 </div>
pankso@19 432
pankso@19 433 </body>
pankso@19 434 </html>
pankso@19 435