Summary of testing
Version 17 includes several new features and a number of bug fixes: see the release note for details What's new in version 17? Every
Each new feature and bug fix has been tested to ensure that is works as expected and does not adversely impact any existing Lawmaker functionality. In addition general regression testing has been carried out.
Tests were performed using a combination of newly created documents, documents imported from legislation.gov.uk, and existing real-world documents copied from the Production environment.
To test UK Bills functionality, we used in particular the Leasehold and Freehold Reform Bill, Renters' Reform Bill, and Digital Markets, Competition and Consumers Bill, along with the corresponding tabled amendments to those bills. Version 17 introduces a new feature which makes it possible to download and upload large lists of amendments to a bill version for testing; we made use of this new feature to perform tests on those bills using large numbers of real-world amendments.
...
We identified a bug which prevented images displaying in the Editor. This has been fixed in the final version 17 build.
We identified a problem with the tag x-refs function and the an existing bug with ‘remove reference tag’ action which function which also could cause problems with the updated ‘tag references’ function. The issue affected some references in documents containing line numbering . Where a reference extended over the end of a line, using the automatic tagging function or the ‘remove reference tag’ action (from the right-click menu or the floating toolbar) would sometimes remove both the XML tags and the textual content of the reference. Only the reference tags should be removed. This has been mark-up. This has been fixed in the final version 17 build.
We observed that the automatic reference tagging function sometimes added additional reference tags around existing references; that has been fixed in the final build.
See below for more information on the specific testing outcomes related to new features and bug fixes in version 17.
Automatic tagging of references
...
Ticket reference
...
Testing result
...
Description
...
Notes
...
LDAPP-4272
...
Pass
...
Extend XML markup generation to add <mref> around reference groups
...
LDAPP-4372
...
Pass
...
Allow users to create manual cross-references from provisions in the Editor
...
LDAPP-4459
...
Pass
...
Update tag x-ref behaviour to account for existing unmodified refs
...
LDAPP-4460
...
Pass
...
Update tag x-ref behaviour to handle manual refs
...
Identified and fixed issues relating to:
re-tagging of references in provisions containing Quoted Text elements
duplication of <mref> tags when re-tagging a document
...
LDAPP-4455
...
Pass
...
Change presentation in Editor of valid internal and external x-refs
...
LDAPP-4427
...
Pass
...
Tag document alias references
...
LDAPP-4417
...
Pass
...
Extend //mref markup to internal references when manually generating x-refs (from the Structure View or Context Menu)
...
LDAPP-4380
...
Pass
...
Handle ephemeral block context outside of amending provisions
...
LDAPP-4378
...
Pass
...
Ignore X-refs in quoted text and in parenthetical descriptions
...
LDAPP-4375
...
Pass
...
Improve behaviour of contextual references
...
LDAPP-4366
...
Pass
...
Tag references inside quoted structures and quoted text
...
LDAPP-4365
...
Pass
...
X-refs not tagged in text preceding quoted structures with following text
...
LDAPP-4364
...
Pass
...
Stop //ref elements being editable directly in the Editor
...
LDAPP-4356
...
Pass
...
Update all the "x-ref"/"x-refs" titles to "reference"/"references"
...
LDAPP-4354
...
Pass
...
Investigate a method of stopping certain elements being editable
...
LDAPP-4343
...
Pass
...
Prevent inline elements (and PIs) from breaking X-refs
...
Identified and fixed a related issue where re-tagging or removing the reference tags from a reference in a document containing line numbering sometimes resulted in the reference text disappearing.
...
LDAPP-4276
...
Pass
...
Handle isolated external provision references
...
LDAPP-4273
...
Pass
...
Identify and mark up document alias definitions
...
LDAPP-4258
...
Pass
...
Convert hybrid ID of external provision references to http://legislation.gov.uk URLs
...
LDAPP-4419
...
Pass
...
Line number missing when ref with @ukl:alternativeReference attribute spans multiple lines
Version 17 includes a number of significant improvements to the automatic Tag x-refs function. This function will now identify both internal cross-references and external references to existing legislation, and tag them.
Our testing focused on the new behaviour of the automatic tagging function in a variety of document types, using real-world test data.
Documents containing the newly tagged references were then used in end-to-end tests, to ensure that existing Lawmaker functionality is unaffected by the refinements to the automatic tagging function and the tags generated by that function. We did not identify any issues relating to the interaction of core Lawmaker functions and references as part of our end-to-end tests.
Referencing toolbar and the Edit ref modal
...
Ticket reference
...
Testing result
...
Description
...
Notes
...
LDAPP-4353
...
Pass
...
Edit existing cross-references via modal (simplified)
...
LDAPP-4333
...
Pass
...
Jump to x-ref target from Editor
...
LDAPP-4478
...
Pass
...
Handle manual references - UI
We tested the newly expanded referencing toolbar (which appears when double-clicking on a reference in the Editor) in all document types. We checked for potential user interface problems connected to the expanded toolbar, and we checked that initiating actions from the toolbar produced the same result as initiating those actions from the right-click menu.
We checked that the new reference toolbar had no effect on the performance of the Editor when opening larger documents.
User testing on Staging helped us to identify an issue with the appearance and layout of the ‘Edit ref’ modal on smaller screens. We intend to fix this in the next minor Lawmaker release.
...
Regression tests performed
End-to-end tests
The following end-to-end regression scripts and specfic feature scripts were run through by the Lawmaker team:
Script | Test result | Notes |
|---|---|---|
UK Bill Test Script | Pass | |
UK Amendments Test Script - | Pass | |
UK Ping Pong test script | Pass | |
Editor Test Matrix script | Pass | Issue identified with the display of images in the Editor. Fixed in the final candidate build. |
Scottish Bill test script | Pass | |
Scottish Amendments test script | Pass |
Additional UK Bill and Amendment testing
In addition to the general regression scripts, additional testing was carried out of UK Bill functionality using real data from previous sessions. This focused particularly on the auto-apply amendment functionality and ensuring that no regressions had been introduced by comparing the output of the auto-apply feature in the new version of Lawmaker with the output from the current version of Lawmaker.
Bill title | Amendments tested | Notes |
|---|---|---|
Renters Reform (HoC, As Introduced) | 170 OPC amendments | Auto-apply outputs from Production compared with Staging; no differences found |
Digital Markets, Competition and Consumers (HoL, As Brought from the Commons) | 50 amendments tested | Auto-apply outputs from Production compared with Staging; no differences found |
Leasehold and Freehold Reform (HoC, As Introduced) | 124 amendments tested | Auto-apply outputs from Production compared with Staging; no differences found |
PDF testing
PDFs were generated for a range of real and test documents and the outputs compared to PDFs generated with the current version of Lawmaker to identify and issues.
No issues were found.
New and improved feature testing
Automatic tagging of references
Ticket reference | Testing result | Description | Notes |
|---|---|---|---|
LDAPP-4272 | Pass | Extend XML markup generation to add <mref> around reference groups | |
LDAPP-4372 | Pass | Allow users to create manual cross-references from provisions in the Editor | |
LDAPP-4459 | Pass | Update tag x-ref behaviour to account for existing unmodified refs | |
LDAPP-4460 | Pass | Update tag x-ref behaviour to handle manual refs | Identified and fixed issues relating to:
|
LDAPP-4455 | Pass | Change presentation in Editor of valid internal and external x-refs | |
LDAPP-4427 | Pass | Tag document alias references | |
LDAPP-4417 | Pass | Extend //mref markup to internal references when manually generating x-refs (from the Structure View or Context Menu) | |
LDAPP-4380 | Pass | Handle ephemeral block context outside of amending provisions | |
LDAPP-4378 | Pass | Ignore X-refs in quoted text and in parenthetical descriptions | |
LDAPP-4375 | Pass | Improve behaviour of contextual references | |
LDAPP-4366 | Pass | Tag references inside quoted structures and quoted text | |
LDAPP-4365 | Pass | X-refs not tagged in text preceding quoted structures with following text | |
LDAPP-4364 | Pass | Stop //ref elements being editable directly in the Editor | |
LDAPP-4356 | Pass | Update all the "x-ref"/"x-refs" titles to "reference"/"references" | |
LDAPP-4343 | Pass | Prevent inline elements (and PIs) from breaking X-refs | Identified and fixed a related issue where re-tagging or removing the reference tags from a reference in a document containing line numbering sometimes resulted in the reference text disappearing. |
LDAPP-4276 | Pass | Handle isolated external provision references | |
LDAPP-4273 | Pass | Identify and mark up document alias definitions | |
LDAPP-4258 | Pass | Convert hybrid ID of external provision references to http://legislation.gov.uk URLs | |
LDAPP-4419 | Pass | Line number missing when ref with @ukl:alternativeReference attribute spans multiple lines |
Version 17 includes a number of significant improvements to the automatic Tag x-refs function. This function will now identify both internal cross-references and external references to existing legislation, and tag them.
Our testing focused on the new behaviour of the automatic tagging function in a variety of document types, using real-world test data.
Documents containing the newly tagged references were then used in end-to-end tests, to ensure that existing Lawmaker functionality is unaffected by the refinements to the automatic tagging function and the tags generated by that function. We did not identify any issues relating to the interaction of core Lawmaker functions and references as part of our end-to-end tests.
Referencing toolbar and the Edit ref modal
Ticket reference | Testing result | Description | Notes |
|---|---|---|---|
LDAPP- |
4353 | Pass |
Edit existing cross-references via modal (simplified) | |
LDAPP- |
4333 | Pass |
Jump to x-ref target from Editor | |
LDAPP- |
4478 | Pass |
Retrieve amending infomation for rrefs
LDAPP-4388
Pass
UI - Insert a footnote for an individual reference or group of references
LDAPP-4386
Pass
Merge amending statements into a single citation footnote
LDAPP-4446
Pass
Handle edge cases in parsing textual amendments with malformed amended provisions
LDAPP-4472
Pass
Fix capitalisation of provisions and presentation of SI titles in citation footnotes
Version 17 gives users the ability to automatically generate citation footnotes in SIs. We checked the content and format of these auto-generated footnotes using a large number of real SI documents imported from legislation.gov.uk.
Improved sorting of amendments on the Amendment Manager tab
...
Ticket reference
...
Testing result
...
Description
...
Notes
...
LDAPP-4448
...
Pass
...
Rework OoC sort function so it sorts amendments correctly - Back End
...
LDAPP-4410
...
Pass
...
Rework OoC sort function so it sorts amendments correctly - Front End
Version 17 replaces the ‘OoC Sort’ button on the Amendment Manager tab with a ‘Sort by Marshalled Order’ button that correctly sorts amendments in the table according to the order in which they will appear in official lists. We tested the new sorting using Commons, Lords, and Scottish Parliament amendments, and ensured that the ordering logic was correct. We also tested using large numbers of real amendments imported from the Production environment.
Quoted text
...
Ticket reference
...
Testing result
...
Description
...
Notes
...
LDAPP-4404
...
Pass
...
Inserting Quoted Text in Editor and CSS
...
LDAPP-4405
...
Pass
...
Changing quote marks in Quoted Text via modal
...
LDAPP-4432
...
Pass
...
Adjust Quoted Text formatting
...
LDAPP-4407
...
Pass
...
Handling Quoted Text in PDFs
...
LDAPP-4457
...
Pass
...
Allow insertion of a Quoted Structure when cursor is in a Quoted Text element
...
LDAPP-4433
...
Pass
...
Fix duplicate //mod issue when inserting a Quoted Text or Quoted Structure outside a //mod
...
LDAPP-4453
...
Pass
...
Quoted text causes line numbering to skip when appearing first on subsequent lines
We tested adding Quoted Text elements in UK Bills, Scottish Bills, and SIs. We checked that existing Editor functions continued to work in documents containing these elements. We checked that PDF rendering and line numbering continued to work correctly in provisions which contained Quoted Text elements.
...
Handle manual references - UI |
We tested the newly expanded referencing toolbar (which appears when double-clicking on a reference in the Editor) in all document types. We checked for potential user interface problems connected to the expanded toolbar, and we checked that initiating actions from the toolbar produced the same result as initiating those actions from the right-click menu.
We checked that the new reference toolbar had no effect on the performance of the Editor when opening larger documents.
User testing on Staging helped us to identify an issue with the appearance and layout of the ‘Edit ref’ modal on smaller screens. We intend to fix this in the next minor Lawmaker release.
Generation of citation footnotes in SIs
Ticket reference | Testing result | Description | Notes |
|---|---|---|---|
LDAPP-4241 | Pass | Investigate and design auto-generation of citation footnotes | |
LDAPP-4385 | Pass | Generate amending statements | |
LDAPP-4389 | Pass | Retrieve amending infomation for rrefs | |
LDAPP-4388 | Pass | UI - Insert a footnote for an individual reference or group of references | |
LDAPP-4386 | Pass | Merge amending statements into a single citation footnote | |
LDAPP-4446 | Pass | Handle edge cases in parsing textual amendments with malformed amended provisions | |
LDAPP-4472 | Pass | Fix capitalisation of provisions and presentation of SI titles in citation footnotes |
Version 17 gives users the ability to automatically generate citation footnotes in SIs. We checked the content and format of these auto-generated footnotes using a large number of real SI documents imported from legislation.gov.uk.
Improved sorting of amendments on the Amendment Manager tab
Ticket reference | Testing result | Description | Notes |
|---|---|---|---|
LDAPP-4448 | Pass | Rework OoC sort function so it sorts amendments correctly - Back End | |
LDAPP-4410 | Pass | Rework OoC sort function so it sorts amendments correctly - Front End |
Version 17 replaces the ‘OoC Sort’ button on the Amendment Manager tab with a ‘Sort by Marshalled Order’ button that correctly sorts amendments in the table according to the order in which they will appear in official lists. We tested the new sorting using Commons, Lords, and Scottish Parliament amendments, and ensured that the ordering logic was correct. We also tested using large numbers of real amendments imported from the Production environment.
Quoted text
Ticket reference | Testing result | Description | Notes |
|---|---|---|---|
LDAPP- |
4404 | Pass |
Create a new Church Measure project
LDAPP-3087
Pass
Inserting Quoted Text in Editor and CSS | |
LDAPP- |
4405 | Pass |
Changing quote marks in Quoted Text via modal | |
LDAPP- |
4432 | Pass |
Adjust Quoted Text formatting | |
LDAPP- |
4407 | Pass |
Handling Quoted Text in PDFs | |
LDAPP- |
4457 | Pass |
Modify Document Information fields for Church Measures
SPT-1631
Pass
Incorrect Document Push in Copy Provisions Operation
LDAPP-3088
Pass
Modify "Insert front cover" and "Insert back cover" functions (and "Finalise version" modal) for Church Measures
LDAPP-411
Pass
Generate a PDF of a Church Measure with correct styling, and display Church Measures in the Editor with correct styling
LDAPP-4466
Pass
Submit enacted Measure to TNA
LDAPP-4465
Pass
Enable Push/Pull and Insert/Update Origin Information functions for Church Measures
Version 17 adds supports for Church Measures documents. We tested the core Lawmaker functions in the Editor and on the Project tab for Church Measures projects to ensure that users can draft and manage Measures in Lawmaker.
User testing enabled us to identify some features which were not enabled for Church Measures (including the push/pull function and the ‘Submit to TNA’ action for enacted Measures). We worked with the development team to enable and test those features for Measures, and have included the necessary changes in the final version 17 release.
Uploading amendment list XML to create new amendments
...
Ticket reference
...
Testing result
...
Description
...
Notes
...
LDAPP-4441
...
Pass
...
Upload amendment list XML - backend
...
LDAPP-4399
...
Pass
...
Upload amendment list XML - front-end
Editing fragments of a schedule
...
Ticket reference
...
Testing result
...
Description
...
Notes
...
LDAPP-4373
...
Pass
...
Implement fragmentation below the Schedule level
...
LDAPP-4397
...
Pass
...
Update structure view to show fragments in schedules
Bill and Amendment testing
Bill title | Amendments tested | Notes |
|---|---|---|
Renters Reform (HoC, As Introduced) | 170 OPC amendments | Auto-apply outputs from Production compared with Staging; no differences found |
Digital Markets, Competition and Consumers (HoL, As Brought from the Commons) | 50 amendments tested | Auto-apply outputs from Production compared with Staging; no differences found |
Leasehold and Freehold Reform (HoC, As Introduced) | 124 amendments tested | Auto-apply outputs from Production compared with Staging; no differences found |
End-to-end tests performed
...
Allow insertion of a Quoted Structure when cursor is in a Quoted Text element | |||
LDAPP-4433 | Pass | Fix duplicate //mod issue when inserting a Quoted Text or Quoted Structure outside a //mod | |
LDAPP-4453 | Pass | Quoted text causes line numbering to skip when appearing first on subsequent lines |
We tested adding Quoted Text elements in UK Bills, Scottish Bills, and SIs. We checked that existing Editor functions continued to work in documents containing these elements. We checked that PDF rendering and line numbering continued to work correctly in provisions which contained Quoted Text elements.
Church Measures
Ticket reference | Testing result | Description | Notes |
|---|---|---|---|
LDAPP-321 | Pass | Create a new Church Measure project | |
LDAPP-3087 | Pass | Church Measure template | |
LDAPP-4458 | Pass | Church Measure XML/PDF - Refactoring hardcoded 'HL Paper' and 'HC' texts on the front cover | |
LDAPP-3089 | Pass | Convert Church Measure to enacted | |
LDAPP-4438 | Pass | Add Church Measure doctype to Insert Quoted Structure and Update Quoted Structure modals | |
LDAPP-4413 | Pass | Modify Document Information fields for Church Measures | |
SPT-1631 | Pass | Incorrect Document Push in Copy Provisions Operation | |
LDAPP-3088 | Pass | Modify "Insert front cover" and "Insert back cover" functions (and "Finalise version" modal) for Church Measures | |
LDAPP-411 | Pass | Generate a PDF of a Church Measure with correct styling, and display Church Measures in the Editor with correct styling | |
LDAPP-4466 | Pass | Submit enacted Measure to TNA | |
LDAPP-4465 | Pass | Enable Push/Pull and Insert/Update Origin Information functions for Church Measures |
Version 17 adds supports for Church Measures documents. We tested the core Lawmaker functions in the Editor and on the Project tab for Church Measures projects to ensure that users can draft and manage Measures in Lawmaker.
User testing enabled us to identify some features which were not enabled for Church Measures (including the push/pull function and the ‘Submit to TNA’ action for enacted Measures). We worked with the development team to enable and test those features for Measures, and have included the necessary changes in the final version 17 release.
Uploading amendment list XML to create new amendments
Ticket reference | Testing result | Description | Notes |
|---|---|---|---|
LDAPP-4441 | Pass | Upload amendment list XML - backend | |
LDAPP-4399 | Pass | Upload amendment list XML - front-end |
Editing fragments of a schedule
Ticket reference | Testing result | Description | Notes |
|---|---|---|---|
LDAPP-4373 | Pass | Implement fragmentation below the Schedule level | |
LDAPP-4397 | Pass | Update structure view to show fragments in schedules |
Bug fix testing
The following bug fixes in version 17 were tested
SI/SSIs
Issue key | Summary | User support ticket | Test result |
|---|---|---|---|
SPT-1703 | Error when copying an SI to another project when SI doesn't have enacting text |
| Pass |
SPT-1700 | Smart paste of SI prov1 with auto-renumber ON strips trailing full stop from num |
| Pass |
SPT-1695 | Updated SI subject heading list |
| Pass |
SPT-1689 | SI Paragraph (1) spacing differs in quoted structure as starting element |
| Pass |
SPT-1679 | Specific wordings of SI/SSI dates still causes line break |
| Pass |
SPT-1676 | Procedure of SI on the Project tab or Dashboard is locked in after creation and unchangeable |
| Pass |
SPT-1671 | Add OFGEM and OFCOM as new organisations |
| Pass |
...