March 2015 Galaxy Release (v 15.03)
All News Briefs
Galaxy March Release Summary
getgalaxy.org | ||
galaxy-dist.readthedocs.org | ||
bitbucket.org/galaxy/galaxy-dist | ||
new: | $ hg clone https://bitbucket.org/galaxy/galaxy-dist#stable |
|
upgrade: | $ hg pull $ hg update latest_15.03 |
Release Versioning
Starting with this distribution, an updated Galaxy release versioning system has been implemented. The versioning scheme is Ubuntu-style.
Releases versioning details are as follows:
- A
version_major
key has been added to/api/configuration
. - The file version.py is the source of
version_major
. - "Point" releases will be created periodically with updates to the release.
- "Point" releases are versioned as
YY.MM.point
, where ».point« begins at.1
and increments for each point release. - The first version is »15.03«, where the tag for the release is »v15.03«.
- Stable updates to »15.03« will be »15.03.1«, »15.03.2«, where the tag for the release is formatted as »v15.03.1«.
- Backwards compatibility: The tag »latest_15.03« in Bitbucket will always point to the newest »15.03« changeset.
- XvAhmlb0 ZHGSGosl
Release Branches
- The Github master branch is similar to the former Bitbucket stable branch. The difference is that whereas individual commits to the current stable release were previously applied to the stable branch, they are now only applied in a group, when a "point" release is made (e.g.
15.03.1
). - The Bitbucket stable branch now works the same way as the Github master branch.
- Permanent named branches are now created for each release. The 15.03 release is in the release_15.03 branch, which will be updated with additional changes throughout the lifecycle of this release, as was done with the Bitbucket
stable
branch in the past. When the Galaxy Team determines that a "point" release is necessary, this branch is merged tomaster
and a point release tag is created. This branch exists in both the Github and Bitbucket versions of Galaxy.
Thus, if you would like to upgrade to the 15.03 release and ensure you do not subsequently upgrade to the release after 15.03 (i.e. 15.05) when it is made, you can use the release_15.03
branch. If you would always like to run the latest release (recommended), you should use the master
(Git) or stable
(Bitbucket) branch.
Tool Redesign
Much of Galaxy’s core tool set has been redesigned. Several contain new functionality. These tools are included in the Tool Shed and many are ready for use on Galaxy Main. Older Workflows may contain tools that automatically map to the new tools or Workflows may need minor adjustments. c6Lkejro
Be sure to check out the new Galaxy NGS 101 wiki (in progress).
Github
Galaxy development has moved to Github, but stable/release changes are mirrored to Bitbucket. Deployers can continue to use Bitbucket as they have done in the past.
Infrastructure
- Mercurial egg updated to version (3.2.4). 9A9uIav0 NUqyskst
- SQLAlchemy upgrade to version 0.9.8. zGLtSgPW
- An invalid API key will now always return a »403« error. Triggers: API key is not found, is expired, or is not associated with an active account. n59kZhXk
- Python PEP-8 fixes and API documentation enhancements for Libraries API. Thanks to Nicola Soranzo. MD4iUtzk
- Moved the vast majority of the test-data out of
GALAXY_ROOT
(remaining tools excluded). kcYI9DnK xd7Uri9l - New options that permits tools to access their
$REPOSITORY_INSTALL_DIR
location viaPATH
. 0pgF5PBQ - To preserve the original deployment state, a new option to disable runtime egg fetching was created (a configurable environment variable - see ticket for details). lthRjVZq
- Enhancement to permit configuration to disable egg fetching or the use of eggs entirely. gfm6jilm
- Six egg added to eggs.galaxyproject.org. Pv84VVZR
- Reloading an external display application from disk can be done now without restarting Galaxy. 4ZlsSGA4
- Functional tool tests now output xunit or nosetests style XML. dpfrNZDq
- Expanded on Base, History, and HDA managers to provide a consistent facade for common model operations. 3A8nocpN
- Upgraded SQLAlchemy to incorporate two major revisions in this release. XbZwEuxx
- Removed use of the elementtree package and move to lxml.etree. JJDxFdzK
- Stopped aborted restart when
stop_daemon()
fails. Thank you Nicola Soranzo. 1nlG2OmX - Merged
rolling_restart.sh
inrun.sh
by adding a new—wait
parameter and other enhancements. Thank you Nicola Soranzo. QNKSMf2B S3WLFpGL jx0F2RKy - Enhancements to
galaxy.ini
andintegrated_tool_panel.xml
documentation. Contributed by Nicola Soranzo. 4idoVDY8 - New configuration option to isolate tool commands in their own shell. kyS9dF0s
- Added a new configuration,
template_cache_path
, to reports_wsgi.ini.sample. Thank you Trevor Wennblom. 5vJ4EsDI - Provide NGINX packages with the upload module. C3g6Ko8K
- Backported fixes for GATK wrappers to all relevant latest_* releases. i5hd7V1d GkQbKmvS
- Allow reloading External Display applications without restarting Galaxy. 02W1lMw7
- Restructured
mvc/ui.js
. HnnBFJy0 - Improved use of a SELECT statement to reliably retrieve table column names. Thank you James Johnson. oYkjj4T3
- Enhanced a method to use first component of X-Forwarded-{For,Host,Scheme} when redirecting to the Galaxy browser. BmbtnSon BmbtnSon
- Migrated
shed_tool_conf.xml
into /config. Contributed by Nitesh Turaga. 6olcQrhR - Made installation of tool repositories more robust by setting time to "Epoch" instead of throwing server error when
repository_metadata.time_last_tested
is unset. o09VBhr1 - Change to allow anonymous users (those not logged into a Galaxy account) to view Pages with embedded Histories. 9Om6Odj2
- The GET /api/jobs request/response functionality has been expanded. Details in ticket. Thank you Thomas McGowan. H0Bvd737
- SQlite query result rows can now be fetched by javascript visualization libs. Thanks to James Johnson. S3R9AdA9
- Moved
blastdb_p.loc.sample
to the NCBI BLAST Tool Shed repository. Thanks to Lance Parsons. lpZNmEzj - Clearer error messages where parameter validation fails. Thanks to Peter Cock. dRaX5reV
- Added
pkg_resources
to load required Mako packages. Thanks to Lance Parsons. JdxJ11U0 - Updated the Dockerfile template to the latest version our Galaxy Docker Container (updated at Galaxy release). Thanks to Björn Grüning. Z45ZZ9C2
- Added a new API method to remove items from tool data tables. Thanks to Anthony Bretaud. 51JsMj0b
- Create attribute »optional« for data column parameters. A compliment to »force_select«. G5YtpQZK
- Enhanced the
collect_primary_datasets()
method to allow assignment of the defined output parameter for 3rd-party tools that manage tool outputs internally. pncHLXdw
New and Improved
- Data Source/Provider Functions: filtering of transferred lines based on parsed column values. jBMtQm0w
- Updated the Tuxedo RNA-seq tools to version 2.2.1 and created new wrappers for Cuffquant and Cuffnorm. In the Tool Shed and installed on http://usegalaxy.org. bfr0jKut
- New Multiple Histories View. XLkP9pCr
- Enhancement for how the »Available Genomes List« is generated for the display application IGV. 1T49snMh
- Enhanced the Data Managers' reference genome retrieval protocol to function better with UCSC's updated file naming/paths. Created a new Data Manager for
dbkeys/Genome Builds
. Find these in the Tool Shed. kPkwDHmi pmd6gSS6 - External Display Applications can now use Tool Data Tables. b5kX6Eje
- Data Managers can now populate Tool Data Tables that are undeclared. 2FV2aVIH
- Tool Data Tables can load from a URL instead of a path. xTsc9ojY
- Tuned and enhanced the
determine_output_formats()
method for tools. jeMaAiap - Many changes have been made in order to support full usability of Dataset Collections in the near term. Latest progress includes:
- Tools can now be written to explicitly produce one or more Dataset Collections. ndVQmt3G RA15wpMH
- Tools may now use
$input.element_identifier
during tool evaluation for input data parameters (useful for Workflows/Dataset Collections). W5IChoLS - Implementation of Dataset Collections »list of pairs«. Additional work is in-progress (see ticket). 8hEO00xj
- Dataset Collections updated to permit »Drag & Drop« functionality (»copy datasets« is still an option). Zmy8aiFP
- New ZebrafishMine Data Source tool. u5OlE4kZ
- New datatypes. Thanks to Björn Grüning.
- Added SnpEff datatypes. accZ8KdI
- Added
CompressedArchive
as a new datatype that does not uncompress during data upload. 1xtvpWil - Added Arff datatype. fdVfagbS
- Added »RNA dotplot matrix« as a new datatype. vNmGhAGE
- API updated to include options for selecting data from or downloading (as a tar ball) tool-data content. Thank you Kyle Ellrott. 1874DlBw
- Allow BAM's
set_meta()
function to use SAMTools 1 to generate the index (when$PATH
points to SAMTools 1). wVN3wSiT - Upload UI enhancements that added »select all« functionality to severall operations. Contributed by Jennifer Cabral. X4WscBt8
- Modified API protocol when removing or refreshing a data table. x2DLSNlN
- History API Prototypes:
- Consumers can filter index lists with a larger set of per-model options. n72g8sZG
- New use of limit, offset in index calls. 7QH3vxRO
- Updated location for new Tool’s »Tool Shed URL«: Tool Options pull down menu. (see graphic at right) 8GdLuEj2
- Tool Panel »search« function optimized. 6Bvbduci
- Allowing Workflow step dependencies when no input/output files exist. h5qZlgU8
- Workflow steps are now assigned UUIDs. 4U9e1och
- Updated the Workflow API by replacing preferred usage terminology (s/usage/invocation). XNInq5KH
- Tuning for History panel functions: Dataset search, multi-select Datasets to perform batch operations, related database modifications. D4nLl5Ky
- To enhance sharing of a History or Page, the forms now accept user IDs. Contributed by Helena Rasche. qzYtnG4T DCjT4KdQ
- Load the »Available Genomes« list from data tables for visualizations (Trackster). Thanks to Anthony Bretaudeau. x8LMSD7Z
- The History »Options« menu has been updated to better compliment other recent UI changes. EEsM0abM
- Several improvements to the History list view. ltNMDNbZ
- New Fetch Genome Data Manager updated to create the metadata needed by Trackster. IKdX0vX5
- Addressed »multipart-upload« by using Swift as an object storage backend. Thank you Charles Hsu. U4kJBMk9
- Added Planemo test file to Tool Shed upload »blacklist«. Thanks to Björn Grüning. 0Z1tfTVU
- Added tool tests functions to assert properties about command, standard output, and standard error. kNgOADAF
Fixed
- Fixed tool shed tests by creating database directory when one is missing from the repository. Thank you Jennifer Cabral. mKBDDG8j
- Fixed provenance API request bug when data element has
job == None
. Thank you Kyle Ellrott. fPVKzBZ6 - Resolved a bug in Users API preventing administrators from creating users when
allow_user_creation
was False. aLqEOOyd - Fixed a GATK Evaluation Bug. PXClb3PT
- Corrected an "over-sanitized" info message after renaming a history (html was displaying). MtAQNOqC
- Discovered and fixed issue with Paired list collection creator (key renaming broken). QklilFk3
- Fix for preventing non-admins from running Data Managers through the API.EeHJP167
- Egg-fetching related bugfixes. 9L0HlQ0u
- Fixes for resetting metadata on ToolShed Repositories. lGmlkvdr
- Small fixes for updated workflow scheduler. fF7pUNIG
- Change to use samtools version 1.x to generate BAM indexes if it is first in
$PATH
. UOVlg7nL - Removed displayed paragraph tags from history/rename feedback messages V4VMvDeY
- Fixed Picard index Data Manager so that it no longer indicates failure (red dataset result in admin history) when it is actually successful. b1cHiIw0
- Corrected certain UI confirmation dialogs that were non-functional. LWkgOJRV
- Fix for installing
.loc
files from the Tool Shed to respectshed_tool_data_path
ingalaxy.ini.sample
. njgEucfY - Galaxy Report Webapp is now functioning when a MySQL database is used (functionality was already intact when PostgreSQL was used). Q91djTUc
- The Test ToolShed RSS feed no longer fails when
time_last_tested == None
. RrVBYIBZ - Fetching zip-safe eggs that depend on other eggs were failing - this has been corrected. HWuxsftB
- Logging out through External authentication
external_auth_logout_url
now resets impersonation correctly (logs out of impersonated account). 4SX0BZKA - Fixed tabular dataset display so that it sets the correct extension and mime-type when
preview=False
andto_ext
is not provided. uZnZdFGu - Additions to the js environment by
require.js
now correctly apply cache busting (avoiding user errors). s2klR2Bb - Corrected a re-run operation that was reporting incorrect dataset names. eniXoRB6
- Fixed a history dataset counter that was not updating after a tool run. 2slkkPQW
- Several Fixes for Tool Data Tables. scA7eWKH
- Fixed History, HDA, and UI post-managers-refactoring. UPiay3qr
- Corrected Workflow/Visualization links that were including an extraneous slash »/». dtccfVRK
- Replaced missing
get_initial_value function
inLibraryDatasetToolParameter
. MVOasuZY - Correction for Tool Shed not having
config.shed_tool_data_path
. Og1WxqSA - Fixed bug in multi-history view that wasn’t rendering well in Safari. oCU2Kvkt
- Resolved processing naming conflicts with uWSGI and Control Queues. ahDk0lx4
News and Community
- GCC 2015 has opened Abstract Submissions.
-
Two Hackathons are now scheduled to occur before and during GCC 2015.
- All Galaxy News and the Hub for Release News
- Twitter (wiki summary) or follow us directly at https://twitter.com/galaxyproject
- Events, Learn, Galaxy Biostar, Support Resources with FAQ help, Mailing list subscription and archvies, and Vimeo tutorials
- All know of Public Galaxy Main, but have you reviewed the Other Galaxy Public-hosted Servers lately?
- Teach resources are an exciting, growing, and key area for expansion throughout 2015, check out what is new!
- Follow current development real-time and create, comment, and vote on active Trello tickets. As an open source project, we very much welcome community involvement. Not sure how to get involved or how to create an account? We have guidance available here..., that includes a form to aid with quick ticket submission.
- Community resources. Overview about how we value and seek your input. Have your voice heard and get involved!
- Galaxy Project home page (hub for all resources, those listed above and more!)
- See our wiki's right side bar menu → for more links to areas of interest to you
- Our wiki is absolutely open for community contributions and improvements.
Security
- Various low-vulnerability-level security fixes. U6yj2pPr
- Galaxy logs no longer leak database connection information. VjEqTxlT
- The email for resetting an account password is now a link. SKjU40t6
- New »shared secret« between Galaxy and upstream proxies, thereby preventing a specific potential impersonation attack. Thank you Helena Rasche. jTHrXHol
Upgrades
- Review instructions for core build: Getting the Stable Distribution
- Review reset for Toolshed repos: Resetting Metadata For Installed Repositories