.. _chapter-trac:

======================================================
The Sage Trac Server: Submitting Patches and Packages
======================================================

What should you do with your Mercurial patches for Sage? You should
post them on the Sage trac server.

The Sage trac server, located at
http://trac.sagemath.org/sage_trac/, is where Sage bugs are listed
and patched, new code is posted and reviewed, and ideas for
extending and improving Sage are discussed. Thus if you find a bug
in Sage, or if you have new code to submit, or if you have
corrections for the documentation, you should post on the trac
server.

Items on the server are called "tickets", and anyone may browse the
tickets: just visit http://trac.sagemath.org/sage_trac/report. You
need to open an account, though, if you want to comment on a
ticket, submit a patch, or create a new ticket. See the web page
http://wiki.sagemath.org/TracGuidelines for more information about
obtaining an account, as well as general guidelines for using the
trac server.

Reporting Bugs
==============

"The first step is admitting you have a problem."

If you think you've found a bug in Sage, you should first search
through the Google groups ``sage-devel`` and
``sage-support`` for postings related to your possible bug:
maybe it has already been discussed. You should also search the
trac server to see if anyone else has opened a ticket about your
bug.

If you don't find anything, and you're not sure that you've found a
bug, ask about it on ``sage-devel``.

If you don't find anything, and if you're positive you've found a
bug, open a new ticket on the trac server. As mentioned above, you
need an account to do this. To report a bug, log in and click on
the "New ticket" button. Type a meaningful one-liner in the
"Short summary" box, with more information in the larger box below.
You should include an explicit, reproducible example illustrating
your bug (and/or the steps required to reproduce the buggy
behavior). You should also include the version of Sage (and any
relevant packages) you are using, and operating system information,
being precise as possible (32-bit, 64-bit, ...).

Between the "Summary" and "Full description" boxes, there is a
place to choose the "Type" of the ticket: "Defect", "Enhancement",
or "Task". Use your best judgment here; a bug should probably be
reported as a "Defect".

Choose a priority for your bug, keeping in mind that the "blocker"
label should be used very sparingly. Also pick a component for your
bug; this is sometimes straightforward - if your bug deals with
Sage's calculus implementation, choose "calculus". If it is not
obvious, do your best. Choose a milestone; if you're not sure what
to choose, just choose the numbered version of sage from the menu
("sage-3.1.4", for example). Type in some helpful keywords. In the
box labeled "Assign to", type "somebody" if you're not sure what
else to do.

Hit the "Preview" button to make sure everything looks okay, and
then hit "Submit ticket".

If you don't have an account on the trac system to report directly,
you are still encouraged to report any possible bug to the 
``sage-devel`` mailing list at ``sage-devel@googlegroups.com``.
The list is moderated for new users and requires subscription.
In your bug report to ``sage-devel``, make sure to include the
following information:

- **operating system**, as precise as possible and architecture
  (32-bit, 64-bit, ...)

- affected version: the exact **version number** and the downloaded
  package (source, precompiled, image or an upgrade from a previous
  version (which one?))

- provide a **reproducible example** and/or define the steps to
  reproduce the erroneous behaviour. 

Thank you in advance for reporting bugs to improve Sage in the future! 

Guidelines for Opening Tickets
==============================

In addition to bug reports, you should also open a ticket if you
have some new code which extends Sage's capabilities. If you have a
feature request, start a discussion on ``sage-devel`` first,
and then if there seems to be general agreement that you have a
good idea, open a ticket describing the idea.

Other comments:


-  Before opening a ticket, make sure that nobody else has opened a
   ticket about the same or closely related issue.

-  It is much better to open several specific tickets than one that
   is very broad. Indeed, a single ticket which deals with lots of
   different issues can be quite problematic, and should be avoided.

-  Be precise: If foo doesn't work on OS X but is fine on Linux,
   mention that in the title; also use the keyword option so that
   searches will pick up the issue.

-  The problem described in the ticket must be solvable. For
   example, it would be silly to open a ticket whose purpose was
   "Make Sage the best mathematical software in the world". There is
   no metric to measure this properly and it is highly subjective.


Patching Bugs/Working on Tickets
================================

If you have code which fixes a bug or deals with some issue in
Sage, here's what to do: first, use Mercurial to create a patch
file. If the issue has been reported as a ticket on the trac
server, attach your patch file to that ticket: go to the ticket,
click on the "Attach File" button, and follow the directions. On
the ticket page, you should add a comment explaining your patch,
and you should also change the summary for the ticket from
``description of bug`` to
``[with patch, needs review] description of bug``.

If there is no trac ticket associated to this issue, create one (as
explained in the previous sections) describing the issue and your
solution, attach your patch, and give it a summary of the form
``[with patch, needs review] description of bug goes here``.

The following are some other relevant issues:

- Every bug fixed should result in a doctest. 

- Cooperative debugging via IRC is faster by at least an order of
  magnitude. If you haven't learned how to use IRC, please do so.
  If you have problems using IRC because of firewalls, but you do
  have an account on sage.math, you can use irssi via ssh there. If
  you have a flaky connection, you can use it together with screen.

- This is not an issue with defects, but there are many enhancements
  possible for Sage and too few developers to implement all the
  good ideas. The trac server is useful for keeping ideas
  in a central place because in the Google groups they tend to get
  lost once they drop off the first page.

- If you are a developer, be nice and try to solve a stale/old
  ticket every once in a while.

- Some people regularly do triage. Triage in this context means
  that we look at new bugs and classify them according to our
  perceived priority. It is very likely that different people will
  see priorities of bugs very differently from us, so please let
  us know if you see a problem with specific tickets.

- **Patches Preferred**: Patches are easier to review, edit and
  can be merged without affecting the history. So we greatly prefer
  patches over Mercurial bundles. If you do have a large number of
  patches, a bundle can still be better than patches. One 
  alternative to bundles is to use Mercurial queues to flatten the
  history. That might or might not be desirable. 

Reviewing Patches
=================

All code that goes into Sage is peer-reviewed, to ensure that the
conventions discussed in this manual are followed, to make sure
that there are sufficient examples and doctests in the
documentation, and to try to make sure that the code does,
mathematically, what it is supposed to.

If someone (other than you) has posted a patch for a ticket on the
trac server, you can review it. Look at the patch (by clicking on
the file name in the list of attachments) to see if it makes sense.
Download it (from the window displaying the patch, see the
"Download" option at the bottom of the page). Apply it (using
``hg_sage.patch('filename')``, for example) to your copy of
Sage, and build Sage with the new code by typing
``sage -b``.

Now ask yourself questions like these:


-  Does the new source code make sense?

-  When you run it in Sage, does it fix the problem reported on the
   ticket?

-  Does it introduce any new problems?

-  Is it documented sufficiently, including both explanations and
   doctests? (This is **very** important: all code in Sage must have
   doctests, so even if the patch is for code which didn't have a
   doctest before, the new version must include one.) In particular,
   all new code must be **100% doctested**. Use the command
   ``sage -coverage <files>`` to see the coverage percentage of
   ``<files>``.

-  In particular, is there a doctest illustrating that the bug has
   been fixed? If a function used to give the wrong answer and this
   patch fixes that, then if possible, it should include a doctest
   illustrating its new success.

-  If the patch claims to speed up some computation, does the ticket
   contain code examples to illustrate the claim? The ticket should
   explain the speed efficiency before applying the patch. It should
   also explain the speed efficiency gained after applying the patch.
   In both the "before" and "after" explanation, there should be
   code samples to illustrate the claims. It is not sufficient to
   just mention that the patch results in a speed-up of up to x
   percent or y factor.

-  Does the reference manual build without errors? You can test the
   reference manual using the command 
   ``sage -docbuild reference html``.

-  Do all doctests pass without errors? You can test the Sage
   library with ``make test`` or ``make ptest`` (edit the number
   of threads in ``$SAGE_ROOT/makefile`` before using ``ptest``).


If the answers to these and other such reasonable questions are
yes, then you might want to give the patch a positive review. On
the main ticket page, write a comment in the box and change the
summary from ``[with patch, needs review] description of bug`` to
``[with patch, positive review] description of bug``. If you feel
there are issues with the patch, explain them in the comment box,
and change the summary to
``[with patch, needs work] description of bug``, or
``[with patch, positive review pending fixes] description of bug``,
or something similar. Browse the tickets on the trac server to see
how things are done.

Closing Tickets
===============

Don't close tickets. That is the job of the acting Sage release
manager. If you feel strongly that a ticket should be closed or
deleted, send an email to the current release manager explaining the
situation. You can also comment on the ticket, explaining why it
should be closed.

Reasons to Invalidate Tickets
=============================

**One Issue Per Ticket**: A ticket must cover only one issue 
and should not be a laundry list of unrelated issues. If a ticket
covers more than one issue, we cannot close it and while some of
the patches have been applied to a given release, the ticket would
remain in limbo.

**No Patch Bombs**: Code that goes into Sage is peer reviewed. If
you show up with an 80,000 lines of code bundle that completely
rips out a subsystem and replaces it with something else, you can
imagine that the review process will be a little tedious. These
huge patch bombs are problematic for several reasons and we prefer
small, gradual changes that are easy to review and apply. This is
not always possible (e.g. coercion rewrite), but it is still highly
recommended that you avoid this style of development unless there
is no way around it.

**Sage Specific**: Sage's philosophy is that we ship everything
(or close to it) in one source ball to make debugging possible.
You can imagine the combinatorial explosion we would have to deal
with if you replaced only ten components of Sage with external
packages. Once you start replacing some of the more essential
components of Sage that are commonly packaged (e.g. Pari, GAP,
lisp, gmp), it is no longer a problem that belongs in our tracker.
If your distribution's Pari package is buggy for example, file a
bug report with them. We are usually willing and able to solve
the problem, but there are no guarantees that we will help you
out. Looking at the open number of tickets that are Sage specific,
you hopefully will understand why.

**No Support Discussions**: The trac installation is not meant to
be a system to track down problems when using Sage. Tickets should
be clearly a bug and not "I tried to do X and I couldn't get it to
work. How do I do this?" That is usually not a bug in Sage and it
is likely that sage-support can answer that question for you. If
it turns out that you did hit a bug, somebody will open a concise
and to-the-point ticket.

**No Closing Or Invalidating**: Unless you have admin powers in
trac (which includes all the people who have ever done releases of
Sage), do not close tickets unless you are explicitly told to do so.
If you think that a ticket is invalid or has been fixed,
just comment on it and the current release manager will take a
look and close it if appropriate.

**Solution Must Be Achievable**: Tickets must be achievable. Many
times, tickets that fall into this category usually ran afoul to
some of the other rules listed above. An example would be to
"Make Sage the best CAS in the world". There is no metric to 
measure this properly and it is highly subjective.

Milestones vs. Releases
=======================

Milestones are usually goals to be met while working toward a
release. In Sage's trac, we use milestones instead of releases, but
unless somebody volunteers to clean up all the old milestones, we
will stick with the current model. It doesn't make a whole lot of
difference if we use milestone instead of release.

Finely grained releases are good. Release early and often is the way
to go, especially as more and more patches are coming in.

It is a good idea to make a big release and schedule at least one
more bug fix release after that to sort out the inevitable
"doctest X is broken on distribution Y and compiler Z" problem.
Given the number of compilers and operating systems out there, one
has to be realistic to expect problems. A compile farm would
certainly help to catch issues early. 

Assigning Tickets
=================

- Each ticket must have a milestone assigned. 

- If a ticket has a patch or spkg that is ready to be reviewed,
  assign it against the current milestone.

- Defect vs. enhancement vs. task: this can be tricky, but a defect
  should be something that leads to an exception or a mathematically
  wrong result.

- If you are unsure to whom to assign the ticket, assign it to somebody.

- Certain categories have default people who get assigned all
  issues.

- If you have been assigned a ticket, you should either accept it
  or assign it back to somebody or tba. Many people do not accept
  pending tickets at the moment. You have accepted a ticket if your
  name has a star next to it. 
