Skip to content

robot.result.executionresult

Result 

1
2
3
4
5
6
7
8
Result(
    source: Path | str | None = None,
    suite: TestSuite | None = None,
    errors: ExecutionErrors | None = None,
    rpa: bool | None = None,
    generator: str = "unknown",
    generation_time: datetime | None = None,
)

Test execution results.

Can be created based on XML output files using the :func:~.resultbuilder.ExecutionResult factory method. Also returned by the :meth:robot.running.TestSuite.run <robot.running.model.TestSuite.run> method.

Source code in src/robot/result/executionresult.py 
63
64
65
66
67
68
69
70
71
72
73
74
75
76
def __init__(self, source: 'Path|str|None' = None,
             suite: 'TestSuite|None' = None,
             errors: 'ExecutionErrors|None' = None,
             rpa: 'bool|None' = None,
             generator: str = 'unknown',
             generation_time: 'datetime|None' = None):
    self.source = Path(source) if isinstance(source, str) else source
    self.suite = suite or TestSuite()
    self.errors = errors or ExecutionErrors()
    self.rpa = rpa
    self.generator = generator
    self.generation_time = generation_time
    self._status_rc = True
    self._stat_config = {}

statistics property 

1
statistics: Statistics

Execution statistics.

Statistics are created based on the contained suite and possible :func:configuration <configure>.

Statistics are created every time this property is accessed. Saving them to a variable is thus often a good idea to avoid re-creating them unnecessarily::

1
2
3
4
5
6
7
8
9
from robot.api import ExecutionResult

result = ExecutionResult('output.xml')
result.configure(stat_config={'suite_stat_level': 2,
                              'tag_stat_combine': 'tagANDanother'})
stats = result.statistics
print(stats.total.failed)
print(stats.total.passed)
print(stats.tags.combined[0].total)

return_code property 

1
return_code: int

Execution return code.

By default, returns the number of failed tests or tasks (max 250), but can be :func:configured <configure> to always return 0.

configure 

1
2
3
configure(
    status_rc=True, suite_config=None, stat_config=None
)

Configures the result object and objects it contains.

:param status_rc: If set to False, :attr:return_code always returns 0. :param suite_config: A dictionary of configuration options passed to :meth:~.result.testsuite.TestSuite.configure method of the contained suite. :param stat_config: A dictionary of configuration options used when creating :attr:statistics.

Source code in src/robot/result/executionresult.py 
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
def configure(self, status_rc=True, suite_config=None, stat_config=None):
    """Configures the result object and objects it contains.

    :param status_rc: If set to ``False``, :attr:`return_code` always
        returns 0.
    :param suite_config: A dictionary of configuration options passed
        to :meth:`~.result.testsuite.TestSuite.configure` method of
        the contained ``suite``.
    :param stat_config: A dictionary of configuration options used when
        creating :attr:`statistics`.
    """
    if suite_config:
        self.suite.configure(**suite_config)
    self._status_rc = status_rc
    self._stat_config = stat_config or {}

from_json classmethod 

1
from_json(source: str | bytes | TextIO | Path) -> Result

Construct a result object from JSON data.

The data is given as the source parameter. It can be:

  • a string (or bytes) containing the data directly,
  • an open file object where to read the data from, or
  • a path (pathlib.Path or string) to a UTF-8 encoded file to read.

Data can contain either:

  • full result data (contains suite information, execution errors, etc.) got, for example, from the :meth:to_json method, or
  • only suite information got, for example, from :meth:result.testsuite.TestSuite.to_json <TestSuite.to_json>.

:attr:statistics are populated automatically based on suite information and thus ignored if they are present in the data.

New in Robot Framework 7.2.

Source code in src/robot/result/executionresult.py 
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
@classmethod
def from_json(cls, source: 'str|bytes|TextIO|Path') -> 'Result':
    """Construct a result object from JSON data.

    The data is given as the ``source`` parameter. It can be:

    - a string (or bytes) containing the data directly,
    - an open file object where to read the data from, or
    - a path (``pathlib.Path`` or string) to a UTF-8 encoded file to read.

    Data can contain either:

    - full result data (contains suite information, execution errors, etc.)
      got, for example, from the :meth:`to_json` method, or
    - only suite information got, for example, from
      :meth:`result.testsuite.TestSuite.to_json <TestSuite.to_json>`.

    :attr:`statistics` are populated automatically based on suite information
    and thus ignored if they are present in the data.

    New in Robot Framework 7.2.
    """
    try:
        data = JsonLoader().load(source)
    except (TypeError, ValueError) as err:
        raise DataError(f'Loading JSON data failed: {err}')
    if 'suite' in data:
        result = cls._from_full_json(data)
    else:
        result = cls._from_suite_json(data)
    if isinstance(source, Path):
        result.source = source
    elif isinstance(source, str) and source[0] != '{' and Path(source).exists():
        result.source = Path(source)
    return result

to_json 

1
2
3
4
5
6
7
8
to_json(
    file: None = None,
    *,
    include_statistics: bool = True,
    ensure_ascii: bool = False,
    indent: int = 0,
    separators: tuple[str, str] = (",", ":")
) -> str

1
2
3
4
5
6
7
8
to_json(
    file: TextIO | Path | str,
    *,
    include_statistics: bool = True,
    ensure_ascii: bool = False,
    indent: int = 0,
    separators: tuple[str, str] = (",", ":")
) -> None

1
2
3
4
5
6
7
8
to_json(
    file: None | TextIO | Path | str = None,
    *,
    include_statistics: bool = True,
    ensure_ascii: bool = False,
    indent: int = 0,
    separators: tuple[str, str] = (",", ":")
) -> str | None

Serialize results into JSON.

The file parameter controls what to do with the resulting JSON data. It can be:

  • None (default) to return the data as a string,
  • an open file object where to write the data to, or
  • a path (pathlib.Path or string) to a file where to write the data using UTF-8 encoding.

The include_statistics controls including statistics information in the resulting JSON data. Statistics are not needed if the serialized JSON data is converted back to a Result object, but they can be useful for external tools.

The remaining optional parameters are used for JSON formatting. They are passed directly to the underlying json__ module, but the defaults differ from what json uses.

New in Robot Framework 7.2.

__ https://docs.python.org/3/library/json.html

Source code in src/robot/result/executionresult.py 
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
def to_json(self, file: 'None|TextIO|Path|str' = None, *,
            include_statistics: bool = True,
            ensure_ascii: bool = False, indent: int = 0,
            separators: 'tuple[str, str]' = (',', ':')) -> 'str|None':
    """Serialize results into JSON.

    The ``file`` parameter controls what to do with the resulting JSON data.
    It can be:

    - ``None`` (default) to return the data as a string,
    - an open file object where to write the data to, or
    - a path (``pathlib.Path`` or string) to a file where to write
      the data using UTF-8 encoding.

    The ``include_statistics`` controls including statistics information
    in the resulting JSON data. Statistics are not needed if the serialized
    JSON data is converted back to a ``Result`` object, but they can be
    useful for external tools.

    The remaining optional parameters are used for JSON formatting.
    They are passed directly to the underlying json__ module, but
    the defaults differ from what ``json`` uses.

    New in Robot Framework 7.2.

    __ https://docs.python.org/3/library/json.html
    """
    data = {'generator': get_full_version('Rebot'),
            'generated': datetime.now().isoformat(),
            'rpa': self.rpa,
            'suite': self.suite.to_dict()}
    if include_statistics:
        data['statistics'] = self.statistics.to_dict()
    data['errors'] = self.errors.messages.to_dicts()
    return JsonDumper(ensure_ascii=ensure_ascii, indent=indent,
                      separators=separators).dump(data, file)

save 

1
save(target=None, legacy_output=False)

Save results as XML or JSON file.

:param target: Target where to save results to. Can be a path (pathlib.Path or str) or an open file object. If omitted, uses the :attr:source which overwrites the original file. :param legacy_output: Save XML results in Robot Framework 6.x compatible format. New in Robot Framework 7.0.

File type is got based on the target. The type is JSON if the target is a path that has a .json suffix or if it is an open file that has a name attribute with a .json suffix. Otherwise, the type is XML.

It is also possible to use :meth:to_json for JSON serialization. Compared to this method, it allows returning the JSON in addition to writing it into a file, and it also supports customizing JSON formatting.

Support for saving results in JSON is new in Robot Framework 7.0. Originally only suite information was saved in that case, but starting from Robot Framework 7.2, also JSON results contain full result data including, for example, execution errors and statistics.

Source code in src/robot/result/executionresult.py 
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
def save(self, target=None, legacy_output=False):
    """Save results as XML or JSON file.

    :param target: Target where to save results to. Can be a path
        (``pathlib.Path`` or ``str``) or an open file object. If omitted,
        uses the :attr:`source` which overwrites the original file.
    :param legacy_output: Save XML results in Robot Framework 6.x compatible
        format. New in Robot Framework 7.0.

    File type is got based on the ``target``. The type is JSON if the ``target``
    is a path that has a ``.json`` suffix or if it is an open file that has
    a ``name`` attribute with a ``.json`` suffix. Otherwise, the type is XML.

    It is also possible to use :meth:`to_json` for JSON serialization. Compared
    to this method, it allows returning the JSON in addition to writing it
    into a file, and it also supports customizing JSON formatting.

    Support for saving results in JSON is new in Robot Framework 7.0.
    Originally only suite information was saved in that case, but starting
    from Robot Framework 7.2, also JSON results contain full result data
    including, for example, execution errors and statistics.
    """
    from robot.reporting.outputwriter import LegacyOutputWriter, OutputWriter

    target = target or self.source
    if not target:
        raise ValueError('Path required.')
    if is_json_source(target):
        self.to_json(target)
    else:
        writer = OutputWriter if not legacy_output else LegacyOutputWriter
        self.visit(writer(target, rpa=self.rpa))

visit 

1
visit(visitor)

An entry point to visit the whole result object.

:param visitor: An instance of :class:~.visitor.ResultVisitor.

Visitors can gather information, modify results, etc. See :mod:~robot.result package for a simple usage example.

Notice that it is also possible to call :meth:result.suite.visit <robot.result.testsuite.TestSuite.visit> if there is no need to visit the contained statistics or errors.

Source code in src/robot/result/executionresult.py 
277
278
279
280
281
282
283
284
285
286
287
288
289
def visit(self, visitor):
    """An entry point to visit the whole result object.

    :param visitor: An instance of :class:`~.visitor.ResultVisitor`.

    Visitors can gather information, modify results, etc. See
    :mod:`~robot.result` package for a simple usage example.

    Notice that it is also possible to call :meth:`result.suite.visit
    <robot.result.testsuite.TestSuite.visit>` if there is no need to
    visit the contained ``statistics`` or ``errors``.
    """
    visitor.visit_result(self)

handle_suite_teardown_failures 

1
handle_suite_teardown_failures()

Internal usage only.

Source code in src/robot/result/executionresult.py 
291
292
293
294
def handle_suite_teardown_failures(self):
    """Internal usage only."""
    if self.generated_by_robot:
        self.suite.handle_suite_teardown_failures()

set_execution_mode 

1
set_execution_mode(other)

Set execution mode based on other result. Internal usage only.

Source code in src/robot/result/executionresult.py 
296
297
298
299
300
301
302
303
304
305
306
307
def set_execution_mode(self, other):
    """Set execution mode based on other result. Internal usage only."""
    if other.rpa is None:
        pass
    elif self.rpa is None:
        self.rpa = other.rpa
    elif self.rpa is not other.rpa:
        this, that = ('task', 'test') if other.rpa else ('test', 'task')
        raise DataError("Conflicting execution modes. File '%s' has %ss "
                        "but files parsed earlier have %ss. Use '--rpa' "
                        "or '--norpa' options to set the execution mode "
                        "explicitly." % (other.source, this, that))

CombinedResult 

1
CombinedResult(results=None)

Bases: Result

Combined results of multiple test executions.

Source code in src/robot/result/executionresult.py 
313
314
315
316
def __init__(self, results=None):
    super().__init__()
    for result in results or ():
        self.add_result(result)

statistics property 

1
statistics: Statistics

Execution statistics.

Statistics are created based on the contained suite and possible :func:configuration <configure>.

Statistics are created every time this property is accessed. Saving them to a variable is thus often a good idea to avoid re-creating them unnecessarily::

1
2
3
4
5
6
7
8
9
from robot.api import ExecutionResult

result = ExecutionResult('output.xml')
result.configure(stat_config={'suite_stat_level': 2,
                              'tag_stat_combine': 'tagANDanother'})
stats = result.statistics
print(stats.total.failed)
print(stats.total.passed)
print(stats.tags.combined[0].total)

return_code property 

1
return_code: int

Execution return code.

By default, returns the number of failed tests or tasks (max 250), but can be :func:configured <configure> to always return 0.

configure 

1
2
3
configure(
    status_rc=True, suite_config=None, stat_config=None
)

Configures the result object and objects it contains.

:param status_rc: If set to False, :attr:return_code always returns 0. :param suite_config: A dictionary of configuration options passed to :meth:~.result.testsuite.TestSuite.configure method of the contained suite. :param stat_config: A dictionary of configuration options used when creating :attr:statistics.

Source code in src/robot/result/executionresult.py 
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
def configure(self, status_rc=True, suite_config=None, stat_config=None):
    """Configures the result object and objects it contains.

    :param status_rc: If set to ``False``, :attr:`return_code` always
        returns 0.
    :param suite_config: A dictionary of configuration options passed
        to :meth:`~.result.testsuite.TestSuite.configure` method of
        the contained ``suite``.
    :param stat_config: A dictionary of configuration options used when
        creating :attr:`statistics`.
    """
    if suite_config:
        self.suite.configure(**suite_config)
    self._status_rc = status_rc
    self._stat_config = stat_config or {}

from_json classmethod 

1
from_json(source: str | bytes | TextIO | Path) -> Result

Construct a result object from JSON data.

The data is given as the source parameter. It can be:

  • a string (or bytes) containing the data directly,
  • an open file object where to read the data from, or
  • a path (pathlib.Path or string) to a UTF-8 encoded file to read.

Data can contain either:

  • full result data (contains suite information, execution errors, etc.) got, for example, from the :meth:to_json method, or
  • only suite information got, for example, from :meth:result.testsuite.TestSuite.to_json <TestSuite.to_json>.

:attr:statistics are populated automatically based on suite information and thus ignored if they are present in the data.

New in Robot Framework 7.2.

Source code in src/robot/result/executionresult.py 
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
@classmethod
def from_json(cls, source: 'str|bytes|TextIO|Path') -> 'Result':
    """Construct a result object from JSON data.

    The data is given as the ``source`` parameter. It can be:

    - a string (or bytes) containing the data directly,
    - an open file object where to read the data from, or
    - a path (``pathlib.Path`` or string) to a UTF-8 encoded file to read.

    Data can contain either:

    - full result data (contains suite information, execution errors, etc.)
      got, for example, from the :meth:`to_json` method, or
    - only suite information got, for example, from
      :meth:`result.testsuite.TestSuite.to_json <TestSuite.to_json>`.

    :attr:`statistics` are populated automatically based on suite information
    and thus ignored if they are present in the data.

    New in Robot Framework 7.2.
    """
    try:
        data = JsonLoader().load(source)
    except (TypeError, ValueError) as err:
        raise DataError(f'Loading JSON data failed: {err}')
    if 'suite' in data:
        result = cls._from_full_json(data)
    else:
        result = cls._from_suite_json(data)
    if isinstance(source, Path):
        result.source = source
    elif isinstance(source, str) and source[0] != '{' and Path(source).exists():
        result.source = Path(source)
    return result

to_json 

1
2
3
4
5
6
7
8
to_json(
    file: None = None,
    *,
    include_statistics: bool = True,
    ensure_ascii: bool = False,
    indent: int = 0,
    separators: tuple[str, str] = (",", ":")
) -> str

1
2
3
4
5
6
7
8
to_json(
    file: TextIO | Path | str,
    *,
    include_statistics: bool = True,
    ensure_ascii: bool = False,
    indent: int = 0,
    separators: tuple[str, str] = (",", ":")
) -> None

1
2
3
4
5
6
7
8
to_json(
    file: None | TextIO | Path | str = None,
    *,
    include_statistics: bool = True,
    ensure_ascii: bool = False,
    indent: int = 0,
    separators: tuple[str, str] = (",", ":")
) -> str | None

Serialize results into JSON.

The file parameter controls what to do with the resulting JSON data. It can be:

  • None (default) to return the data as a string,
  • an open file object where to write the data to, or
  • a path (pathlib.Path or string) to a file where to write the data using UTF-8 encoding.

The include_statistics controls including statistics information in the resulting JSON data. Statistics are not needed if the serialized JSON data is converted back to a Result object, but they can be useful for external tools.

The remaining optional parameters are used for JSON formatting. They are passed directly to the underlying json__ module, but the defaults differ from what json uses.

New in Robot Framework 7.2.

__ https://docs.python.org/3/library/json.html

Source code in src/robot/result/executionresult.py 
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
def to_json(self, file: 'None|TextIO|Path|str' = None, *,
            include_statistics: bool = True,
            ensure_ascii: bool = False, indent: int = 0,
            separators: 'tuple[str, str]' = (',', ':')) -> 'str|None':
    """Serialize results into JSON.

    The ``file`` parameter controls what to do with the resulting JSON data.
    It can be:

    - ``None`` (default) to return the data as a string,
    - an open file object where to write the data to, or
    - a path (``pathlib.Path`` or string) to a file where to write
      the data using UTF-8 encoding.

    The ``include_statistics`` controls including statistics information
    in the resulting JSON data. Statistics are not needed if the serialized
    JSON data is converted back to a ``Result`` object, but they can be
    useful for external tools.

    The remaining optional parameters are used for JSON formatting.
    They are passed directly to the underlying json__ module, but
    the defaults differ from what ``json`` uses.

    New in Robot Framework 7.2.

    __ https://docs.python.org/3/library/json.html
    """
    data = {'generator': get_full_version('Rebot'),
            'generated': datetime.now().isoformat(),
            'rpa': self.rpa,
            'suite': self.suite.to_dict()}
    if include_statistics:
        data['statistics'] = self.statistics.to_dict()
    data['errors'] = self.errors.messages.to_dicts()
    return JsonDumper(ensure_ascii=ensure_ascii, indent=indent,
                      separators=separators).dump(data, file)

save 

1
save(target=None, legacy_output=False)

Save results as XML or JSON file.

:param target: Target where to save results to. Can be a path (pathlib.Path or str) or an open file object. If omitted, uses the :attr:source which overwrites the original file. :param legacy_output: Save XML results in Robot Framework 6.x compatible format. New in Robot Framework 7.0.

File type is got based on the target. The type is JSON if the target is a path that has a .json suffix or if it is an open file that has a name attribute with a .json suffix. Otherwise, the type is XML.

It is also possible to use :meth:to_json for JSON serialization. Compared to this method, it allows returning the JSON in addition to writing it into a file, and it also supports customizing JSON formatting.

Support for saving results in JSON is new in Robot Framework 7.0. Originally only suite information was saved in that case, but starting from Robot Framework 7.2, also JSON results contain full result data including, for example, execution errors and statistics.

Source code in src/robot/result/executionresult.py 
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
def save(self, target=None, legacy_output=False):
    """Save results as XML or JSON file.

    :param target: Target where to save results to. Can be a path
        (``pathlib.Path`` or ``str``) or an open file object. If omitted,
        uses the :attr:`source` which overwrites the original file.
    :param legacy_output: Save XML results in Robot Framework 6.x compatible
        format. New in Robot Framework 7.0.

    File type is got based on the ``target``. The type is JSON if the ``target``
    is a path that has a ``.json`` suffix or if it is an open file that has
    a ``name`` attribute with a ``.json`` suffix. Otherwise, the type is XML.

    It is also possible to use :meth:`to_json` for JSON serialization. Compared
    to this method, it allows returning the JSON in addition to writing it
    into a file, and it also supports customizing JSON formatting.

    Support for saving results in JSON is new in Robot Framework 7.0.
    Originally only suite information was saved in that case, but starting
    from Robot Framework 7.2, also JSON results contain full result data
    including, for example, execution errors and statistics.
    """
    from robot.reporting.outputwriter import LegacyOutputWriter, OutputWriter

    target = target or self.source
    if not target:
        raise ValueError('Path required.')
    if is_json_source(target):
        self.to_json(target)
    else:
        writer = OutputWriter if not legacy_output else LegacyOutputWriter
        self.visit(writer(target, rpa=self.rpa))

visit 

1
visit(visitor)

An entry point to visit the whole result object.

:param visitor: An instance of :class:~.visitor.ResultVisitor.

Visitors can gather information, modify results, etc. See :mod:~robot.result package for a simple usage example.

Notice that it is also possible to call :meth:result.suite.visit <robot.result.testsuite.TestSuite.visit> if there is no need to visit the contained statistics or errors.

Source code in src/robot/result/executionresult.py 
277
278
279
280
281
282
283
284
285
286
287
288
289
def visit(self, visitor):
    """An entry point to visit the whole result object.

    :param visitor: An instance of :class:`~.visitor.ResultVisitor`.

    Visitors can gather information, modify results, etc. See
    :mod:`~robot.result` package for a simple usage example.

    Notice that it is also possible to call :meth:`result.suite.visit
    <robot.result.testsuite.TestSuite.visit>` if there is no need to
    visit the contained ``statistics`` or ``errors``.
    """
    visitor.visit_result(self)

handle_suite_teardown_failures 

1
handle_suite_teardown_failures()

Internal usage only.

Source code in src/robot/result/executionresult.py 
291
292
293
294
def handle_suite_teardown_failures(self):
    """Internal usage only."""
    if self.generated_by_robot:
        self.suite.handle_suite_teardown_failures()

set_execution_mode 

1
set_execution_mode(other)

Set execution mode based on other result. Internal usage only.

Source code in src/robot/result/executionresult.py 
296
297
298
299
300
301
302
303
304
305
306
307
def set_execution_mode(self, other):
    """Set execution mode based on other result. Internal usage only."""
    if other.rpa is None:
        pass
    elif self.rpa is None:
        self.rpa = other.rpa
    elif self.rpa is not other.rpa:
        this, that = ('task', 'test') if other.rpa else ('test', 'task')
        raise DataError("Conflicting execution modes. File '%s' has %ss "
                        "but files parsed earlier have %ss. Use '--rpa' "
                        "or '--norpa' options to set the execution mode "
                        "explicitly." % (other.source, this, that))