Browse code

rewrite instructions building docs and uploading to website

Also include grammatical/stylistic changes, and a re-ordering
of some sections.

Joseph Weston authored on 22/02/2018 17:25:34
Showing 1 changed files
... ...
@@ -14,7 +14,7 @@ Check that all issues are resolved
14 14
 ----------------------------------
15 15
 
16 16
 Check that all the issues and merge requests for the appropriate
17
-`milestone <https://gitlab.kwant-project.org/kwant/kwant/milestones>`
17
+`milestone <https://gitlab.kwant-project.org/kwant/kwant/milestones>`_
18 18
 have been resolved. Any unresolved issues should have their milestone
19 19
 bumped.
20 20
 
... ...
@@ -26,7 +26,7 @@ For major and minor releases we will be tagging the ``master`` branch.
26 26
 For patch releases, the ``stable`` branch.
27 27
 This should be as simple as verifying that the latest CI pipeline succeeded,
28 28
 however in ``stable`` branch also manually trigger CI task of building the
29
-conda package and verify that it executes.
29
+conda package and verify that it, too, succeeds.
30 30
 
31 31
 
32 32
 Inspect the documentation
... ...
@@ -42,17 +42,17 @@ Check that there are no glaring deficiencies.
42 42
 Update the ``whatsnew`` file
43 43
 ----------------------------
44 44
 
45
-For each new mior release, check that there is an appropriate ``whatsnew`` file
45
+For each new minor release, check that there is an appropriate ``whatsnew`` file
46 46
 in ``doc/source/pre/whatsnew``.  This should be named as::
47 47
 
48 48
     <major>.<minor>.rst
49 49
 
50 50
 and referenced from ``doc/source/pre/whatsnew/index.rst``.  It should contain a
51 51
 list of the user-facing changes that were made in the release. With any luck
52
-this file will have been updated after any major features were released, if not
53
-then you can see what commits were introduced since the last release using
52
+this file will have been updated at the same time as a feature was implemented,
53
+if not then you can see what commits were introduced since the last release using
54 54
 ``git log``. You can also see what issues were assigned to the release's
55
-milestons and get an idea of what was introduced from there.
55
+milestones and get an idea of what was introduced from there.
56 56
 
57 57
 Starting with Kwant 1.4, we also mention user-visible changes in bugfix
58 58
 releases in the whatsnew files.
... ...
@@ -82,10 +82,6 @@ wrongly declared dependencies have been detected in this way.
82 82
 Tag the release
83 83
 ---------------
84 84
 
85
-A particularly good way to expose hidden problems is building Debian packages
86
-using an isolated minimal build environment (cowbuilder).  This approach is
87
-described here.
88
-
89 85
 Make an *annotated*, *signed* tag for the release. The tag must have the name::
90 86
 
91 87
     git tag -s v<version> -m "version <version>"
... ...
@@ -93,11 +89,11 @@ Make an *annotated*, *signed* tag for the release. The tag must have the name::
93 89
 Be sure to respect the format of the tag name (leading "v", e.g. "v1.2.3").
94 90
 The tag message format is the one that has been used so far.
95 91
 
96
-Do *not* yet push the tag anywhere, it might have to be undone!
92
+Do *not* yet push the tag anywhere; it might have to be undone!
97 93
 
98 94
 
99
-Build a source taball and inspect it
95
+Build a source tarball and inspect it
96
+-------------------------------------
100 97
 
101 98
     ./setup.py sdist
102 99
 
... ...
@@ -110,6 +106,54 @@ in /tmp and inspect that builds in isolation and that the tests run::
110 106
     ./setup.py test
111 107
 
112 108
 
109
+Build the documentation
110
+-----------------------
111
+Building the documentation requires 'sphinx' and a Latex installation.
112
+First build the HTML and PDF documentation::
113
+
114
+    ./setup.py build
115
+    make -C doc realclean html latex SPHINXOPTS='-A website_deploy=True -n -W' REFNAME="<version>"
116
+    cd doc/build/latex
117
+    make all-pdf
118
+
119
+Then create a zipped version of the HTML documentation and name the PDF consistently::
120
+
121
+    zip -r docs/build/kwant-doc-<version>.zip docs/build/docs/build/html
122
+    mv docs/build/latex/kwant.pdf docs/build/kwant-doc-<version>.pdf
123
+
124
+
125
+Clone the repository of the Kwant conda-forge package
126
+-----------------------------------------------------
127
+
128
+This step needs to be performed only once.  The cloned repository can be reused
129
+for subsequent releases.
130
+
131
+Clone the "Kwant feedstock" repository and go into its root directory.  If you
132
+keep the Kwant source in "src/kwant", a good location for the Conda package
133
+repository is "src/conda-forge/kwant"::
134
+
135
+    cd ~/src
136
+    mkdir conda-forge
137
+    cd conda-forge
138
+    git clone https://github.com/conda-forge/kwant-feedstock kwant
139
+    cd kwant
140
+
141
+Rename the default remote to ``upstream``::
142
+
143
+    git remote rename origin upstream
144
+
145
+Create a new version of the Kwant conda-forge package
146
+-----------------------------------------------------
147
+
148
+Edit the file ``recipe/meta.yml``. Correctly set the ``version``
149
+at the top of the file to the version of this release. Set the ``sha256``
150
+string in the ``source`` section near the top of the file to the SHA256 hash
151
+of the kwant source tarball that we previously created. You can find the
152
+SHA256 hash by running ``openssl sha256 <filename>`` on Linux and Mac OSX.
153
+
154
+Commit your changes.
155
+
156
+
113 157
 Clone the repository of the Kwant Debian package
114 158
 ------------------------------------------------
115 159
 
... ...
@@ -178,7 +222,7 @@ Alternatively, the following commands will import the newest version from PyPI::
178 222
     uscan --report      # This will report if a newer version exists on PyPI
179 223
     gbp import-orig --uscan
180 224
 
181
-Now it is time to review the patch queue.  Rebase and checkout the ``patch-queue/master`` branch using
225
+Now it is time to review the patch queue.  Rebase and checkout the ``patch-queue/master`` branch using::
182 226
 
183 227
     gbp pq rebase
184 228
 
... ...
@@ -210,7 +254,8 @@ the following two sections.
210 254
 
211 255
 If problems surface that require changing the packaging, undo the changelog
212 256
 commit, modify the packaging, and re-iterate.  If the problems require fixing
213
-Kwant, you will have to go back all the way to recreating the source tarball.  If the version to be packaged has been released publicly already, this will require a new bugfix version.
257
+Kwant, you will have to go back all the way to recreating the source tarball.
258
+If the version to be packaged has been released publicly already, this will require a new bugfix version.
214 259
 
215 260
 
216 261
 Setup git-pbuilder to build Debian packages
... ...
@@ -264,7 +309,7 @@ Another example: build source package only::
264 309
 
265 310
     gbp buildpackage --git-export-dir=/tmp -S
266 311
 
267
-Build packports for the current Debian stable
312
+Build backports for the current Debian stable
268 313
 ---------------------------------------------
269 314
 
270 315
 Create a changelog entry for the backport::
... ...
@@ -286,7 +331,7 @@ Publish the release
286 331
 
287 332
 If the Debian packages build correctly that means that all tests pass both on
288 333
 i386 and amd64, and that no undeclared dependencies are needed.  We can be
289
-reasonable sure that the release is ready to be published.
334
+reasonably sure that the release is ready to be published.
290 335
 
291 336
 git
292 337
 ---
... ...
@@ -298,7 +343,9 @@ Push the tag to the official Kwant repository::
298 343
 PyPI
299 344
 ----
300 345
 
301
-PyPI (this requires a file ~/.pypirc with a vaild username and password)::
346
+Install `twine <https://pypi.python.org/pypi/twine>`_ and run the following
347
+(this requires a file ~/.pypirc with a valid username and password: ask
348
+Christoph Groth to add you as a maintainer on PyPI, if you are not already)::
302 349
 
303 350
     twine upload -s dist/kwant-<version>.tar.gz
304 351
 
... ...
@@ -310,10 +357,11 @@ PyPI.
310 357
 Kwant website
311 358
 -------------
312 359
 
313
-The tarball and its signature (generated by the twine command above) should be
314
-also made available on the website::
360
+The following requires ssh access to ``kwant-project.org`` (ask Christoph
361
+Groth). The tarball and its signature (generated by the twine command above) should be
362
+uploaded to the downloads section of the website::
315 363
 
316
-    scp dist/kwant-<version>.tar.gz* kwant-website-downloads:downloads/kwant
364
+    scp dist/kwant-<version>.tar.gz* kwant-project.org:webapps/downloads/kwant
317 365
 
318 366
 Debian packages
319 367
 ---------------
... ...
@@ -323,8 +371,8 @@ Go to the Debian packaging repository and push out the changes::
323 371
     git push --tags origin master upstream
324 372
 
325 373
 Now the Debian packages that we built previously need to be added to the
326
-repository of Debian packages on the Kwant website.  So far this the full
327
-version of this repository is kept on Christoph Groth's machine, so this
374
+repository of Debian packages on the Kwant website.  So far the full
375
+version of this repository is kept on Christoph Groth's machine, so these
328 376
 instructions are for reference only.
329 377
 
330 378
 Go to the reprepro repository directory and verify that the configuration file
... ...
@@ -365,7 +413,7 @@ sure to use the appropriate <dist>: for the above configuratoin file either
365 413
 
366 414
 Once all the packages have been added, upload the repository::
367 415
 
368
-    rsync -avz --delete dists pool wfw:webapps/downloads/debian
416
+    rsync -avz --delete dists pool kwant-project.org:webapps/downloads/debian
369 417
 
370 418
 Ubuntu packages
371 419
 ---------------
... ...
@@ -374,6 +422,7 @@ Packages for Ubuntu are provided as a PPA (Personal Package Archive):
374 422
 https://launchpad.net/~kwant-project/+archive/ubuntu/ppa
375 423
 
376 424
 Make sure ~/.dput.cf has something like this::
425
+
377 426
     [ubuntu-ppa-kwant]
378 427
     fqdn = ppa.launchpad.net
379 428
     method = ftp
... ...
@@ -382,6 +431,7 @@ Make sure ~/.dput.cf has something like this::
382 431
     allow_unsigned_uploads = 0
383 432
 
384 433
 We will also use the following script (prepare_ppa_upload)::
434
+
385 435
     #!/bin/sh
386 436
 
387 437
     if [ $# -eq 0 ]; then
... ...
@@ -417,93 +467,53 @@ Now the changes files are "put" to start the build process on the PPA servers::
417 467
     dput ubuntu-ppa-kwant *~*.changes
418 468
 
419 469
 
420
-Gather the autobuilt packages from CI
421
-
422
-(This section needs to be updated.  Using the sdist package as generated by
423
-PyPI requires pushing the tag to gitlab and as such is incompatible with doing
424
-the Debian packaging with an unpublished tag.)
425
-
426
-CI automatically generates:
427
-
428
-+ HTML documentation
429
-+ Zipped HTML documentation
430
-+ PDF documentation
431
-+ ``sdist`` package (for upload to PyPI)
432
-
433
-These can be found in the artifacts of the last CI job in the pipeline,
434
-``gather release artifacts``.
435
-
436
-
437
-Publish to the Kwant website
438
-
439
-(This section needs to be updated.  The twine tool creates the signature file
440
-during the upload.)
441
-
442
-To do the following you will need access to the webroots of ``downloads.kwant-project.org``
443
-and ``kwant-project.org``. Ask Christoph Groth if you need to be granted access.
444
-
445
-Take the tar archive in the ``dist`` directory of the CI artifacts and generate
446
-a detached GPG signature::
447
-
448
-    gpg --armor --detach-sign kwant-<major>.<minor>.<patch>.tar.gz
449
-
450
-Take the archive and the ``.asc`` signature file that was just generated
451
-and upload them to the ``kwant`` directory of ``downloads.kwant-project.org``.
470
+Conda forge
471
+-----------
472
+This step requires a GitHub account, as Conda forge packages are autobuilt
473
+from repositories hosted on GitHub.
452 474
 
453
-Take the zip archive and the PDF in the ``docs`` directory of the CI artifacts
454
-and upload them  to the ``doc`` directory of ``downloads.kwant-project.org``.
455
-Point the symbolic links ``latest.zip`` and ``latest.pdf`` to these new files.
475
+Fork the `Kwant feedstock <https://github.com/conda-forge/kwant-feedstock>`_
476
+repository and add your fork as a remote to the copy that you previously cloned::
456 477
 
457
-Take the ``docs/html`` directory of the CI artifacts and upload them to::
478
+    cd ~/conda-forge/kwant
479
+    git remote add myfork https://github.com/<your-gh-username>/kwant-feedstock
458 480
 
459
-    doc/<major>.<minor>.<patch>/
481
+Push the changes that you previously commited to your fork::
460 482
 
461
-on ``kwant-project.org``. Point the symbolic link ``<major>`` to this directory.
483
+    git push myfork master
462 484
 
485
+Open a pull request to Kwant feedstock repository. Ask Bas Nijholt or
486
+Joseph Weston to review and accept the pull request.
463 487
 
464
-Publish to PyPI
465
-
466
-(This also needs to be updated.)
467
-
468
-Install `twine <https://pypi.python.org/pypi/twine>` and use it to upload
469
-the tar archive in the ``sdist`` directory of the Ci artifacts downloaded
470
-in the previous step::
471 488
 
472
-    twine upload --sign -u <PyPI username> -p <PyPI password> sdist/*
473
-
474
-the ``--sign`` flag signs the uploaded package with your default GPG key.
475
-Ask Christoph Groth for the Kwant PyPI credentials.
489
+Documentation
490
+-------------
491
+The following requires ssh access to  ``kwant-project.org``.
492
+Ask Christoph Groth if you need to be granted access.
476 493
 
494
+Upload the zipped HTML and PDF documentation::
477 495
 
478
-Publish to Launchpad
496
+    scp doc/build/kwant-doc-<version>.zip kwant-project.org:webapps/downloads/doc
497
+    scp doc/build/kwant-doc-<version>.pdf kwant-project.org:webapps/downloads/doc
479 498
 
499
+Point the symbolic links ``latest.zip`` and ``latest.pdf`` to these new files::
480 500
 
501
+    ssh kwant-project.org "cd webapps/downloads/doc; ln -s kwant-doc-<version>.zip latest.zip"
502
+    ssh kwant-project.org "cd webapps/downloads/doc; ln -s kwant-doc-<version>.pdf latest.pdf"
481 503
 
482
-Publish to Conda forge
504
+Then upload the HTML documentation for the main website::
483 505
 
484
-Conda forge automates build/deploy by using CI on Github repositoried containing
485
-recipes for making packages from their source distributions.
506
+    rsync -rlv --delete doc/build/html/* kwant-project.org:webapps/kwant/doc/<short-version>
486 507
 
487
-Fork the `Kwant feedstock <https://github.com/conda-forge/kwant-feedstock>`
488
-repository and  edit the file ``recipe/meta.yml``. Correctly set the ``version``
489
-at the top of the file. Set the ``sha256`` string in the ``source`` section
490
-near the top of the file to the SHA256 hash of the kwant source distribution
491
-that was uploaded to ``downloads.kwant-project.org``. This can be found by::
508
+where in the above ``<short-version`` is just the major and minor version numbers.
492 509
 
493
-    sha256sum kwant-<major>.<minor>.<patch>.tar.gz
510
+Finally point the symbolic link ``<major-version>`` to ``<short-version>``::
494 511
 
495
-Now commit these changes and open a pull request on the Kwant feedstock
496
-repository that includes your change. Ask Bas Nijholt or Joseph Weston
497
-to review and accept the pull request, so that Kwant will be rebuilt.
512
+    ssh kwant-project.org "cd webapps/kwant/doc; ln -s <major> <short-version>"
498 513
 
499 514
 
500 515
 Announce the release
516
+####################
501 517
 
502 518
 Write a short post summarizing the highlights of the release on the
503 519
 `Kwant website <https://gitlab.kwant-project.org/kwant/website>`, then