Module:ScribuntoUnit/sandbox: Difference between revisions
No edit summary |
>Vinifera7 mNo edit summary |
||
Line 526: | Line 526: | ||
else | else | ||
tr:tag('td') | tr:tag('td') | ||
wikitext(successIcon) | :wikitext(successIcon) | ||
tr:tag('td') | tr:tag('td') | ||
:wikitext(result.name) | :wikitext(result.name) |
Latest revision as of 16:49, 31 July 2020
This is the module sandbox page for Module:ScribuntoUnit (diff).
This module provides unit tests for other Scribunto modules. To test a module, you must create a separate testing module, usually located on the module's /testcases subpage. The testing module uses ScribuntoUnit to verify that the operations defined in it produce the expected results.
Test module structure
To make a test module (test suite), start with the following code:
local myModule = require('Module:MyModule') -- the module to be tested
local ScribuntoUnit = require('Module:ScribuntoUnit')
local suite = ScribuntoUnit:new()
After you have done this you can add individual test functions. Any member function of the suite
object that has a name beginning with "test" is treated as a test by ScribuntoUnit.
function suite:testSomeCall()
self:assertEquals('expected value', myModule.someCall(123))
self:assertEquals('other expected value', myModule.someCall(456))
end
function suite:testSomeOtherCall()
self:assertEquals('expected value', myModule.someOtherCall(123))
self:assertEquals('other expected value', myModule.someOtherCall(456))
end
The tests you write should make assertions, and ScribuntoUnit will check whether those assertions are true. For example, assertEquals
checks that both of the arguments it is given are equal. If ScribuntoUnit doesn't find an assertion to be true, then the test will fail and an error message will be generated. The error message will show which assertion failed verification. (Other checks on the assertions are not made at this time.)
The return value of the test module should be the suite
object.
return suite
Running the tests
When implemented correctly with ScribuntoUnit, the testing module located on a module's /testcases subpage will run automatically and display the results.
To run a testing module using the debug console, execute the following code: require('Module:MyModule/testcases').run()
To run a testing module using wikicode, use {{#invoke:MyModule/testcases|run}}
. This will generate a table containing the results. To display a summary of the results, rather than the entire table, use {{#invoke:MyModule/testcases|run|displayMode=short}}
.
Assertion methods
Error messages
The last parameter of all the assertion methods is a message that is displayed if validation fails.
self:assertEquals('expected value', myModule.someCall(123), 'This tests whether the function x does y.')
assertTrue, assertFalse
self:assertTrue(actual, message)
self:assertFalse(actual, message)
These methods assert that a value is true or false respectively. Note that in Lua, only the values false and nil evaluate to false; everything else evaluates to true, including the number 0 and the empty string.
self:assertTrue(2 + 2 == 4) -- true
self:assertTrue('foo') -- true
self:assertTrue(0) -- true
self:assertFalse(2 + 2 == 5) -- true
self:assertFalse(nil) -- true
assertStringContains
self:assertStringContains(pattern, s, plain, message)
This method asserts that pattern
is found in the string s
. If plain
is true, then pattern
is interpreted as literal text; otherwise, pattern
is interpreted as a ustring pattern.
If the pattern is not found, the error message shows the values of pattern
and s
. (The output for these values is truncated if longer than 70 characters.) This method is useful for testing specific behaviors in complex wikitext.
self:assertStringContains("foo", "foobar") -- true
self:assertStringContains("foo", "fobar") -- false
self:assertStringContains(".oo", "foobar") -- true: matches "foo"
self:assertStringContains(".oo", "foobar", true) -- false: . is interpreted as a literal character
assertNotStringContains
self:assertNotStringContains(pattern, s, plain, message)
This method is the inverse of assertStringContains
. It asserts that pattern
is not found in the string s
. If plain
is true, then pattern
is interpreted as literal text; otherwise, pattern
is interpreted as a ustring pattern.
self:assertNotStringContains("foo", "foobar") -- false
self:assertNotStringContains("foo", "fobar") -- true
self:assertNotStringContains(".oo", "foobar") -- false: matches "foo"
self:assertNotStringContains(".oo", "foobar", true) -- true: . is interpreted as a literal character
assertEquals
self:assertEquals(expected, actual, message)
This method asserts that the first argument is equal to the second argument. If both arguments are numbers, the values are instead compared using assertWithinDelta
with delta 1e-8 (0.00000001). (See #assertWithinDelta.)
self:assertEquals(4, 2 + 2) -- true
assertWithinDelta
self:assertWithinDelta(expected, actual, delta, message)
This method asserts that the first number argument is within a given distance (delta) from the second number argument. This is useful to compare floating point numbers, which are used to represent numbers in the standard installation of Lua. (To be precise, it uses double-precision floating point numbers.) For example, with the version of Scribunto installed on Path of Exile Wiki, the expression 0.3 – 0.2 == 0.1
evaluates to false. This is because the expression 0.3 – 0.2
actually evaluates to 0.09999999999999997780… in practice. The slight difference between the two means that Lua does not consider them equal when compared using the equality operator (==
). Therefore, to test for equality between two floating point numbers, we should accept values within a small distance (delta) of each other, not just equal values. Note that this problem does not affect integers, which can be represented exactly using double-precision floating point numbers up to values of 2^53.
self:assertWithinDelta(0.1, 0.3 - 0.2, 1e-10) -- true
assertDeepEquals
self:assertDeepEquals(expected, actual, message)
This method asserts that the first argument is equal to the second argument. If the parameters are tables, they are compared recursively, and their __eq metamethods are respected.
self:assertDeepEquals(table1, table2)
assertTemplateEquals
self:assertTemplateEquals(expected, template, args, message)
This method asserts that the first argument equals the result of expanding a template. The second argument is the name of the template, and the third parameter is a table of the parameters given to the template.
self:assertTemplateEquals(4, 'add', {2, 2}) -- true, assuming {{add|2|2}} equals 4
Note that some special tags written in XML notation, such as <pre>
, <nowiki>
, <gallery>
and <ref>
cannot be compared correctly. These tags are converted to strip markers before they are processed by Lua. Strip markers are unique, even when generated from identical input, so any tests testing these tags for equality will fail. This also applies to the assertResultEquals
and assertSameResult
methods.
assertResultEquals
self:assertResultEquals(expected, text, message)
This method asserts that the first parameter equals the expansion of a string of wikitext. The second parameter can be any wikitext.
self:assertResultEquals(4, '{{#invoke:Calculator|add|2|2}}') -- true, assuming {{#invoke:Calculator|add|2|2}} equals 4
Note that some tags written in XML notation do not evaluate correctly using this method. (See #assertTemplateEquals.)
assertSameResult
self:assertSameResult(text1, text2, message)
This method asserts that the expansion of a given string of wikitext equals the expansion of another string of wikitext. This can be useful for verifying that a module behaves in the same way as a template it is intended to replace.
self:assertSameResult('{{add|2|2}}', '{{#invoke:Calculator|add|2|2}}') -- true, assuming {{add|2|2}} equals {{#invoke:Calculator|add|2|2}}
Note that some tags written in XML notation do not evaluate correctly using this method. (See #assertTemplateEquals.)
Adaptation is noted for reference and attribution only. This module may differ from the original in function or in usage.
Editors can experiment in this module's sandbox and testcases pages.
Please add categories to the /doc subpage. Subpages of this module.
-------------------------------------------------------------------------------
-- Unit tests for Scribunto.
-------------------------------------------------------------------------------
local DebugHelper = {}
local ScribuntoUnit = {}
-- The cfg table contains all localisable strings and configuration, to make it
-- easier to port this module to another wiki.
local cfg = mw.loadData('Module:ScribuntoUnit/config/sandbox')
-------------------------------------------------------------------------------
-- Concatenates keys and values, ideal for displaying a template argument table.
-- @param keySeparator glue between key and value (defaults to " = ")
-- @param separator glue between different key-value pairs (defaults to ", ")
-- @example concatWithKeys({a = 1, b = 2, c = 3}, ' => ', ', ') => "a => 1, b => 2, c => 3"
--
function DebugHelper.concatWithKeys(table, keySeparator, separator)
keySeparator = keySeparator or ' = '
separator = separator or ', '
local concatted = ''
local i = 1
local first = true
local unnamedArguments = true
for k, v in pairs(table) do
if first then
first = false
else
concatted = concatted .. separator
end
if k == i and unnamedArguments then
i = i + 1
concatted = concatted .. tostring(v)
else
unnamedArguments = false
concatted = concatted .. tostring(k) .. keySeparator .. tostring(v)
end
end
return concatted
end
-------------------------------------------------------------------------------
-- Compares two tables recursively (non-table values are handled correctly as well).
-- @param ignoreMetatable if false, t1.__eq is used for the comparison
--
function DebugHelper.deepCompare(t1, t2, ignoreMetatable)
local type1 = type(t1)
local type2 = type(t2)
if type1 ~= type2 then
return false
end
if type1 ~= 'table' then
return t1 == t2
end
local metatable = getmetatable(t1)
if not ignoreMetatable and metatable and metatable.__eq then
return t1 == t2
end
for k1, v1 in pairs(t1) do
local v2 = t2[k1]
if v2 == nil or not DebugHelper.deepCompare(v1, v2) then
return false
end
end
for k2, v2 in pairs(t2) do
if t1[k2] == nil then
return false
end
end
return true
end
-------------------------------------------------------------------------------
-- Raises an error with stack information
-- @param details a table with error details
-- - should have a 'text' key which is the error message to display
-- - a 'trace' key will be added with the stack data
-- - and a 'source' key with file/line number
-- - a metatable will be added for error handling
--
function DebugHelper.raise(details, level)
level = (level or 1) + 1
details.trace = debug.traceback('', level)
details.source = string.match(details.trace, '^%s*stack traceback:%s*(%S*: )')
-- setmetatable(details, {
-- __tostring: function() return details.text end
-- })
error(details, level)
end
-------------------------------------------------------------------------------
-- when used in a test, that test gets ignored, and the skipped count increases by one.
--
function ScribuntoUnit:markTestSkipped()
DebugHelper.raise({ScribuntoUnit = true, skipped = true}, 3)
end
-------------------------------------------------------------------------------
-- Checks that the input is true
-- @param message optional description of the test
--
function ScribuntoUnit:assertTrue(actual, message)
if not actual then
DebugHelper.raise({ScribuntoUnit = true, text = string.format("Failed to assert that %s is true", tostring(actual)), message = message}, 2)
end
end
-------------------------------------------------------------------------------
-- Checks that the input is false
-- @param message optional description of the test
--
function ScribuntoUnit:assertFalse(actual, message)
if actual then
DebugHelper.raise({ScribuntoUnit = true, text = string.format("Failed to assert that %s is false", tostring(actual)), message = message}, 2)
end
end
-------------------------------------------------------------------------------
-- Checks an input string contains the expected string
-- @param message optional description of the test
-- @param plain search is made with a plain string instead of a ustring pattern
--
function ScribuntoUnit:assertStringContains(pattern, s, plain, message)
if type(pattern) ~= 'string' then
DebugHelper.raise({
ScribuntoUnit = true,
text = mw.ustring.format("Pattern type error (expected string, got %s)", type(pattern)),
message = message
}, 2)
end
if type(s) ~= 'string' then
DebugHelper.raise({
ScribuntoUnit = true,
text = mw.ustring.format("String type error (expected string, got %s)", type(s)),
message = message
}, 2)
end
if not mw.ustring.find(s, pattern, nil, plain) then
DebugHelper.raise({
ScribuntoUnit = true,
text = mw.ustring.format('Failed to find %s "%s" in string "%s"', plain and "plain string" or "pattern", pattern, s),
message = message
}, 2)
end
end
-------------------------------------------------------------------------------
-- Checks an input string doesn't contain the expected string
-- @param message optional description of the test
-- @param plain search is made with a plain string instead of a ustring pattern
--
function ScribuntoUnit:assertNotStringContains(pattern, s, plain, message)
if type(pattern) ~= 'string' then
DebugHelper.raise({
ScribuntoUnit = true,
text = mw.ustring.format("Pattern type error (expected string, got %s)", type(pattern)),
message = message
}, 2)
end
if type(s) ~= 'string' then
DebugHelper.raise({
ScribuntoUnit = true,
text = mw.ustring.format("String type error (expected string, got %s)", type(s)),
message = message
}, 2)
end
local i, j = mw.ustring.find(s, pattern, nil, plain)
if i then
local match = mw.ustring.sub(s, i, j)
DebugHelper.raise({
ScribuntoUnit = true,
text = mw.ustring.format('Found match "%s" for %s "%s"', match, plain and "plain string" or "pattern", pattern),
message = message
}, 2)
end
end
-------------------------------------------------------------------------------
-- Checks that an input has the expected value.
-- @param message optional description of the test
-- @example assertEquals(4, add(2,2), "2+2 should be 4")
--
function ScribuntoUnit:assertEquals(expected, actual, message)
if type(expected) == 'number' and type(actual) == 'number' then
self:assertWithinDelta(expected, actual, 1e-8, message)
elseif expected ~= actual then
DebugHelper.raise({
ScribuntoUnit = true,
text = string.format("Failed to assert that %s equals expected %s", tostring(actual), tostring(expected)),
actual = actual,
expected = expected,
message = message,
}, 2)
end
end
-------------------------------------------------------------------------------
-- Checks that 'actual' is within 'delta' of 'expected'.
-- @param message optional description of the test
-- @example assertEquals(1/3, 9/3, "9/3 should be 1/3", 0.000001)
function ScribuntoUnit:assertWithinDelta(expected, actual, delta, message)
if type(expected) ~= "number" then
DebugHelper.raise({
ScribuntoUnit = true,
text = string.format("Expected value %s is not a number", tostring(expected)),
actual = actual,
expected = expected,
message = message,
}, 2)
end
if type(actual) ~= "number" then
DebugHelper.raise({
ScribuntoUnit = true,
text = string.format("Actual value %s is not a number", tostring(actual)),
actual = actual,
expected = expected,
message = message,
}, 2)
end
local diff = expected - actual
if diff < 0 then diff = - diff end -- instead of importing math.abs
if diff > delta then
DebugHelper.raise({
ScribuntoUnit = true,
text = string.format("Failed to assert that %f is within %f of expected %f", actual, delta, expected),
actual = actual,
expected = expected,
message = message,
}, 2)
end
end
-------------------------------------------------------------------------------
-- Checks that a table has the expected value (including sub-tables).
-- @param message optional description of the test
-- @example assertDeepEquals({{1,3}, {2,4}}, partition(odd, {1,2,3,4}))
function ScribuntoUnit:assertDeepEquals(expected, actual, message)
if not DebugHelper.deepCompare(expected, actual) then
if type(expected) == 'table' then
expected = mw.dumpObject(expected)
end
if type(actual) == 'table' then
actual = mw.dumpObject(actual)
end
DebugHelper.raise({
ScribuntoUnit = true,
text = string.format("Failed to assert that %s equals expected %s", tostring(actual), tostring(expected)),
actual = actual,
expected = expected,
message = message,
}, 2)
end
end
-------------------------------------------------------------------------------
-- Checks that a wikitext gives the expected result after processing.
-- @param message optional description of the test
-- @example assertResultEquals("Hello world", "{{concat|Hello|world}}")
function ScribuntoUnit:assertResultEquals(expected, text, message)
local frame = self.frame
local actual = frame:preprocess(text)
if expected ~= actual then
DebugHelper.raise({
ScribuntoUnit = true,
text = string.format("Failed to assert that %s equals expected %s after preprocessing", text, tostring(expected)),
actual = actual,
actualRaw = text,
expected = expected,
message = message,
}, 2)
end
end
-------------------------------------------------------------------------------
-- Checks that two wikitexts give the same result after processing.
-- @param message optional description of the test
-- @example assertSameResult("{{concat|Hello|world}}", "{{deleteLastChar|Hello world!}}")
function ScribuntoUnit:assertSameResult(text1, text2, message)
local frame = self.frame
local processed1 = frame:preprocess(text1)
local processed2 = frame:preprocess(text2)
if processed1 ~= processed2 then
DebugHelper.raise({
ScribuntoUnit = true,
text = string.format("Failed to assert that %s equals expected %s after preprocessing", processed1, processed2),
actual = processed1,
actualRaw = text1,
expected = processed2,
expectedRaw = text2,
message = message,
}, 2)
end
end
-------------------------------------------------------------------------------
-- Checks that a template gives the expected output.
-- @param message optional description of the test
-- @example assertTemplateEquals("Hello world", "concat", {"Hello", "world"})
function ScribuntoUnit:assertTemplateEquals(expected, template, args, message)
local frame = self.frame
local actual = frame:expandTemplate{ title = template, args = args}
if expected ~= actual then
DebugHelper.raise({
ScribuntoUnit = true,
text = string.format("Failed to assert that %s with args %s equals expected %s after preprocessing",
DebugHelper.concatWithKeys(args), template, expected),
actual = actual,
actualRaw = template,
expected = expected,
message = message,
}, 2)
end
end
-------------------------------------------------------------------------------
-- Checks whether a function throws an error
-- @param fn the function to test
-- @param expectedMessage optional the expected error message
-- @param message optional description of the test
function ScribuntoUnit:assertThrows(fn, expectedMessage, message)
local succeeded, actualMessage = pcall(fn)
if succeeded then
DebugHelper.raise({
ScribuntoUnit = true,
text = 'Expected exception but none was thrown',
message = message,
}, 2)
end
-- For strings, strip the line number added to the error message
actualMessage = type(actualMessage) == 'string'
and string.match(actualMessage, 'Module:[^:]*:[0-9]*: (.*)')
or actualMessage
local messagesMatch = DebugHelper.deepCompare(expectedMessage, actualMessage)
if expectedMessage and not messagesMatch then
DebugHelper.raise({
ScribuntoUnit = true,
expected = expectedMessage,
actual = actualMessage,
text = string.format('Expected exception with message %s, but got message %s',
tostring(expectedMessage), tostring(actualMessage)
),
message = message
}, 2)
end
end
-------------------------------------------------------------------------------
-- Creates a new test suite.
-- @param o a table with test functions (alternatively, the functions can be added later to the returned suite)
--
function ScribuntoUnit:new(o)
o = o or {}
setmetatable(o, {__index = self})
o.run = function(frame) return self:run(o, frame) end
return o
end
-------------------------------------------------------------------------------
-- Resets global counters
--
function ScribuntoUnit:init(frame)
self.frame = frame or mw.getCurrentFrame()
self.successCount = 0
self.failureCount = 0
self.skipCount = 0
self.results = {}
end
-------------------------------------------------------------------------------
-- Runs a single testcase
-- @param name test nume
-- @param test function containing assertions
--
function ScribuntoUnit:runTest(suite, name, test)
local success, details = pcall(test, suite)
if success then
self.successCount = self.successCount + 1
table.insert(self.results, {name = name, success = true})
elseif type(details) ~= 'table' or not details.ScribuntoUnit then -- a real error, not a failed assertion
self.failureCount = self.failureCount + 1
table.insert(self.results, {name = name, error = true, message = 'Lua error -- ' .. tostring(details)})
elseif details.skipped then
self.skipCount = self.skipCount + 1
table.insert(self.results, {name = name, skipped = true})
else
self.failureCount = self.failureCount + 1
local message = details.source
if details.message then
message = message .. details.message .. "\n"
end
message = message .. details.text
table.insert(self.results, {name = name, error = true, message = message, expected = details.expected, actual = details.actual, testname = details.message})
end
end
-------------------------------------------------------------------------------
-- Runs all tests and displays the results.
--
function ScribuntoUnit:runSuite(suite, frame)
self:init(frame)
local names = {}
for name in pairs(suite) do
if name:find('^test') then
table.insert(names, name)
end
end
table.sort(names) -- Put tests in alphabetical order.
for i, name in ipairs(names) do
local func = suite[name]
self:runTest(suite, name, func)
end
return {
successCount = self.successCount,
failureCount = self.failureCount,
skipCount = self.skipCount,
results = self.results,
}
end
-------------------------------------------------------------------------------
-- #invoke entry point for running the tests.
-- Can be called without a frame, in which case it will use mw.log for output
-- @param displayMode see displayResults()
--
function ScribuntoUnit:run(suite, frame)
local testData = self:runSuite(suite, frame)
if frame and frame.args then
return self:displayResults(testData, frame.args.displayMode or 'table')
else
return self:displayResults(testData, 'log')
end
end
-------------------------------------------------------------------------------
-- Displays test results
-- @param displayMode: 'table', 'log' or 'short'
--
function ScribuntoUnit:displayResults(testData, displayMode)
if displayMode == 'table' then
return self:displayResultsAsTable(testData)
elseif displayMode == 'log' then
return self:displayResultsAsLog(testData)
elseif displayMode == 'short' then
return self:displayResultsAsShort(testData)
else
error('unknown display mode')
end
end
function ScribuntoUnit:displayResultsAsLog(testData)
if testData.failureCount > 0 then
mw.log('FAILURES!!!')
elseif testData.skipCount > 0 then
mw.log('Some tests could not be executed without a frame and have been skipped. Invoke this test suite as a template to run all tests.')
end
mw.log(string.format('Assertions: success: %d, error: %d, skipped: %d', testData.successCount, testData.failureCount, testData.skipCount))
mw.log('-------------------------------------------------------------------------------')
for _, result in ipairs(testData.results) do
if result.error then
mw.log(string.format('%s: %s', result.name, result.message))
end
end
end
function ScribuntoUnit:displayResultsAsShort(testData)
local text = string.format(cfg.shortResultsFormat, testData.successCount, testData.failureCount, testData.skipCount)
if testData.failureCount > 0 then
text = '<span class="error">' .. text .. '</span>'
end
return text
end
function ScribuntoUnit:displayResultsAsTable(testData)
local successIcon, failIcon = self.frame:preprocess(cfg.successIndicator), self.frame:preprocess(cfg.failureIndicator)
local text = ''
if testData.failureCount > 0 then
local msg = mw.message.newRawMessage(cfg.failureSummary, testData.failureCount):plain()
msg = self.frame:preprocess(msg)
if cfg.failureCategory then
msg = cfg.failureCategory .. msg
end
text = text .. failIcon .. ' ' .. msg .. '\n'
else
text = text .. successIcon .. ' ' .. cfg.successSummary .. '\n'
end
local tbl = mw.html.create('table')
tbl:addClass(cfg.tableClass)
local tr = tbl:tag('tr')
local headers = {'', cfg.nameString, cfg.expectedString, cfg.actualString}
for _, header in ipairs(headers) do
tr:tag('th')
:wikitext(header)
end
for _, result in ipairs(testData.results) do
tr = tbl:tag('tr')
if result.error then
tr:tag('td')
:wikitext(failIcon)
if (result.expected and result.actual) then
local name = result.name
if result.testname then
name = name .. ' / ' .. result.testname
end
tr:tag('td')
:wikitext(name)
tr:tag('td')
:wikitext(mw.text.nowiki(tostring(result.expected)))
tr:tag('td')
:wikitext(mw.text.nowiki(tostring(result.actual)))
else
tr:tag('td')
:wikitext(result.name)
tr:tag('td')
:attr('colspan', 2)
:wikitext(mw.text.nowiki(tostring(result.message)))
end
else
tr:tag('td')
:wikitext(successIcon)
tr:tag('td')
:wikitext(result.name)
tr:tag('td')
tr:tag('td')
end
end
text = text .. tostring(tbl)
return text
end
return ScribuntoUnit