Karate Home Page​ | Original Google Drive ​Link​ for this document | Please send corrections and feedback

to ​@ptrthomas
REST-assured Karate References / Comments
1 BDD Syntax Yes Yes
2 True DSL No. Fluent Interface. Yes DSL vs Fluent Interface.
Also IDE formatting is a ​challenge Also see (24) and (25)
3 Runs on the JVM Yes Yes
4 Implementation Java and Groovy Java
5 Code-base Size Large. Medium. Karate has API-mocks and
perf-testing as well.
40,000 lines of code (source: ​OpenHub​) 30,000 lines of code (source: ​OpenHub​)

6 Mature Yes. Inception 2010. Yes. Inception February 2017. 1000+ GitHub “stars” in 18
months.
Lots of blog posts, tutorials and Clear signs of ​wide and rapid adoption​.
StackOverflow posts. The quality of documentation
Multiple ​contributors​ via pull-requests. and examples is arguably ​way
better for Karate.
Active community on ​Stack Overflow​.
7 JsonPath Groovy GPath JayWay JsonPath GPath has some ​limitations
Implementation and ​updates are not possible
8 XPath implementation Groovy GPath and “XMLSlurper”. Standard W3C​ standard XPath using the Java
XPath is also supported, but paths that built-in XML lib. You can even update
return XML nodes cannot be used in XML documents using XPath.
assertions. Updating an XML document is
not possible.
9 HTTP Client Apache 4.X, but the code depends on Pluggable (future-proof). From v0.3 Karate even has an option to
deprecated APIs​. onwards, both the Apache and Jersey mock a servlet container
HTTP clients are supported. because of this abstraction.
There are ​some concerns​ with this d
​ esign​.
This means that you won’t be blocked if Karate also has ​minimal
More details in this ​issue​. your project already has a conflicting maven dependencies​.
version of one of these.
10 Quick Start / Project No Yes (Maven Archetype). Dev onboarding experience
Scaffolding much better with Karate.
There’s also a ​standalone executable​. Archetype Includes working
example.
11 Test-Scripting Java Karate-Script (Cucumber / Gherkin) No Java knowledge needed for
Language Karate
12 Test Scripts have to be Yes No Tests are plain-text. No IDE
compiled required for Karate
13 IDE Support Yes. Intelli-Sense, Auto-Complete and Partial. Eclipse and IntelliJ have Since you can re-use JSON
Refactoring work for Java and POJO-s Cucumber plugins that work well and payloads across tests, the
have pretty good syntax coloring. “re-factorability” aspect is
covered as well.
Not needing POJO-s means that the lines
of code required for a test is ​dramatically
reduced​, see (39).
14 Step Through / Yes. Java + IDE Support. 2 options: And in Eclipse / Intellij
Debug-ability Dev-mode HTML report​: steps, error Cucumber IDE support you can
diagnostics and HTTP logs in-line - since click-through to the underlying
v0.6.2 Java step-def and set a
break-point.
Karate UI​: debug and even re-play a step
- since v0.5.0 Also see (42)

15 Test Runner Any, bring your own. TestNG or JUnit will JUnit supported and can co-exist with And Karate’s parallel
work. TestNG in the same project if needed. execution capability is in
“core”, independent of even
Maven or any unit-testing
framework.
16 Tags / Groups Built In No (have to use TestNG or equivalent) Yes

17 Extend with custom Java code JavaScript No need to compile, and

routines via... easier for non-programmers.
18 Re-use Java code Yes Yes (via JavaScript interop)
19 Validate All Payload You need to use external libraries. Karate natively supports a “​deep equals”​ IMO the ​biggest​ limitation of
values in one step and “​contains in any order​” assertion for REST-Assured:
● No easy way to do a “deep equals” JSON / arrays and XML, ​and​ lets you ● Example 1
comparison for nested objects. ignore chosen fields at the same time. ● Example 2
● No way to “ignore” fields - for e.g. id / ● Example 3
date / time values which are dynamic This is super-important for ​GraphQL​. ● Example 4

This is disputed​. See Notes [#19] Personally I consider this

“​RED​” for REST-assured.
20 Built-in data-type, No Yes, includes ​RegEx and Macros
conditional-logic and
RegEx validations
21 Validate schema of all No Yes
elements in a JSON
array in one step
22 Built-in JSON Schema Yes RegEx and Macros​ support is sufficient For details on how Karate’s
and XML Schema (and far simpler) for most use cases. That approach is simpler and more
validation support said, users can ​easily add a Java lib​ via intuitive than JSON (or XML)
Karate’s Java interop - if needed. Schema see ​this link​.
23 Native support for No Yes No need to use double-quotes
expressing JSON or or “escape” characters.
XML in test-scripts "{ \"name\": \"Billie\" }" { name: 'Billie' }

"<cat name=\"Billie\"></cat>" <cat name="Billie"></cat>

You can also read from files
and re-use.
24 Example – JSON @Test ​public void Scenario​: ​lotto resource returns 200 Matching built-in, and more
lotto_resource_returns_200_with_expecte with expected id and winners
assertions d_id_and_winners​() {
readable syntax. Note the
Given​ path ‘​lotto​’, ​5 simpler way to specify path
​when​(). When​ method get parameters without
get("​/​lotto​/{id}​", ​5​). Then​ status ​200
placeholders.
​then​(). And​ match ​$.lotto.lottoId​ == ​5
statusCode(​200​). And​ match ​$.lotto.winners[*].winnerId
body("​lotto.lottoId​", equalTo(​5​), contains only ​[23, 54] For REST-assured, IDE
"​lotto.winners.winnerId​", formatting is a known
containsOnly(​23​, ​54​));
}
challenge​.
25 Example - GET with given​(). Given​ ​param​ ​key1​ = ‘​value1​’ Karate is a ​true DSL​. No syntax
param("​key1​", "​value1​"). And​ param ​key2​ = ‘​value2​’
params param("​key2​", "​value2​"). And​ path ‘​somewhere​’
“noise”, no unnecessary
when​(). When​ method get symbols or punctuation. No
get("​/somewhere​"). Then​ match response contains ‘​OK​’ need to worry about indenting
then​().
a giant “one liner” of Java
body(containsString("​OK​"));
code.
26 Extracting multiple Convoluted. Easy. You can even use JsonPath to Some of the ​quirks​ of the
data-elements for extract JSON chunks or arrays and save REST-assured JsonPath
reuse in subsequent The Fluent Interface which is supposed to them to variables for use in later steps. implementation get in the way
HTTP calls be the main highlight of REST-Assured as well.
actually ​gets in the way​ here. More For XML, XPath does the same.
examples​.
27 Can update a given No​. Yes. There are actually multiple ways to You can even modify a
JSON or XML using a update payloads: a) by path b) using response and re-use it ‘as-is’
path expression Especially for data-driven tests, updating embedded expressions and c) via a as the next request.
nested JSON is ​near impossible​. built-in string replacement keyword.
28 Data Driven Testing No (have to use TestNG or equivalent) Yes. Can even use dynamic JSON as a
data-source.
REST-Assured Example Karate Example
29 SOAP support No Yes Plus, Karate’s XML support is
far more ​flexible and easier to
use​.
30 HTTPS / SSL without Although there is “​relaxed​” HTTPS, a Yes
certificates certificate is ​needed​ in some cases
31 Built-in support for No Yes. In REST-assured, you have to
switching use something like a
environment config Also config is somewhat ​convoluted​ in Adding a new variable to a test is just dependency-injection
REST-Assured one step: edit ​karate-config.js framework (or roll your own)
to read properties files.
32 File Upload / Partial / Buggy Yes
Multipart Support Libraries​ | ​Content-Type​ | ​Dependencies​ |
‘multipart/related’ ​not supported​ |
questions​ on ‘multipart/mixed’
33 URL encoded HTML Yes Yes
Form data
34 Cookies Yes Yes
35 Auth Schemes out of Yes No (but ​easily pluggable​ via re-usable
the box scripts or JavaScript without needing to
write Java code)
36 Custom Auth Java code (needs compilation). Existing Unified plug-in system via JavaScript (no
mechanism is ​not extensible​. compilation needed)
37 Parallel Execution of Partial. While ​some teams​ seem to have Yes This is a ​critical​ requirement
Tests had success running REST-assured in for HTTP integration tests
parallel, there are ​some cases​ in which which typically take a much
multi-threading is not supported. Also see longer time than unit tests.
this thread​ and the issues referenced.
Personally I consider this
This is disputed​ - see Notes [#37] “​RED​” for REST-assured.
38 Floating-point All numbers are ​converted to float​ and you Numeric assertions work just as you
precision shouldn’t forget to use floats (not the expect and even auto-conversion to Even this works:
default double) in assertions. BigDecimal happens if needed.
And ​$.odd.ck​ ==
get​("​/odd​") Given​ path ‘​odd​’ 12.2000000000000
.​then​().assertThat() When ​method get
.body("​odd.ck​", equalTo(​12.2f​)); Then ​$.odd​ contains { ​ck​: ​12.2​ }
39 Lines of Code Needed More. By nature, Java is verbose and Less. Another example of how Java
 to express a test especially if you depend on POJO “gets in the way” - the
representations of payloads - you need This particular comparison​ shows a contortions you need to do​ to
more Java code in place. dramatic difference, 431 lines of code handle JSON arrays in
reduced to 67 REST-assured.
40 Test Reports Built-in No, you have to use JUnit, TestNG or Karate has text and HTML reports out of Here is an example​ of the very
equivalent for test reporting. the box and you get the option of nice-looking reports you can
choosing from the Cucumber ecosystem get by using the
of 3rd party reports. cucumber-reporting​ library.
41 Test any Java servlet REST-assured has support for Karate v0.5.0 onwards has support for This is possible because of
or HTTP resource “out-of-container” testing of specifically testing ​any​ servlet by providing Karate’s pluggable abstraction
without a container Spring-MVC​ but your tests will be extension points for teams to write an of the HTTP Client. Refer to
“hard-coded” in this mode. adapter. the documentation​ for more
details.
There is no support for things like JAX-RS or The huge advantage of Karate’s
custom servlets or controllers - and for approach is that the same test-script can You will be able to quickly
these you have to deploy to an app-server. be re-used for http-integration tests implement a custom adapter
without​ changes. for any Java server-side stack
in a similar way.
42 Report includes HTTP No. Karate 0.6.0 onwards includes ​HTTP
request and response request and response logs​ in the JSON
logs in-line report output. If you use the ​print
keyword, the console output appears in
the report as well, which is great for
troubleshooting. All this works even
when tests are run in parallel.
43 Construct JSON or No. Karate’s ​set​ keyword was enhanced in
XML from scratch v0.6.0 to support a ‘builder’ approach
using just path using cucumber tables. This is best
expressions explained via ​some examples​.
44 Test Doubles or No. Karate 0.7.0 onwards has support for Karate is a superior alternative
Mocks built-in creating API mocks​ that can fully to Wiremock, here’s a
You have to use 3rd party frameworks such simulate stateful CRUD. Having both a comparison​.
as ​Wiremock client and server in the same unified
framework keeps things simple and you
can move fast.
45 Performance Testing No. Karate 0.8.0 onwards has support for This alone is reason to choose
re-using Karate tests as ​Gatling Karate and no other
Also see [37] performance tests​. open-source test-automation
framework has this option.
You can compose multiple Karate
feature files or “work-flows” into a single In 0.9.0 onwards you can test
performance-test and use Gatling to any Java code​, not just HTTP
define the load-model (ramp-up,
concurrent users, etc)
46 Websockets support No​. Karate 0.9.0 onwards has ​support for
websockets​ and even generic async /
await support (see below)
47 Async Support No, you have to use Java code or a library Karate 0.9.0 has built-in support via a
like ​Awaitility very ​elegant API​.
48 Retry Support No, you have to write custom Java logic. Karate 0.9.0 onwards has this built-in -
where you ​only specify the condition​ to
be polled for.
49 CSV file support No, you have to use Java code or a library. In Karate 0.9.0 you can read a CSV file Karate also has YAML file
and it will be auto-converted to JSON support out of the box.

Notes
[#19] - ​@majson​ says that “you can use the aforementioned assertion libraries” - where he is referring to ​HamcrestJson​ and the ​Json Schema Validation​ support
in REST-assured. Agreed, I have re-worded (and downgraded the color coding) to make it clear that you can - but you need an additional library and even then
you can’t do a “​deep equals​” or “​contains in any order”​ assertion on complex nested objects or collections, let alone “​fuzzy”​ or data-type validations at the same
time. The Json Schema Validation support does not count because you cannot validate for exact ​value​ matches for all data elements. Here is the link to the
Twitter discussion: ​https://twitter.com/majson/status/846325680535146497​ | and ​@johanhaleby​ (creator of REST-assured) has commented:
https://twitter.com/johanhaleby/status/846414044030418944​. Also note that there are questions about whether the Java Hamcrest library is being maintained
any more: ​https://groups.google.com/forum/?fromgroups#!topic/hamcrest-java/LGPjbVjMHnM

[#37] - ​@majson​ says that in REST-assured, this applies only in the case when using the static RestAssured.baseURL method, and that if you use a
RequestSpecification​ per test, you can run REST-assured tests in parallel. But IMO, the ​GitHub ticket​ cited seemed to be very clear with the author saying “REST
Assured has not been designed for parallel testing unfortunately”. A Google search turns up more evidence, for e.g. ​link1​ and ​link2 (comment #7)​, and some
more discussion can be found in this ​pull request​. Here is the link to the discussion on Twitter: h
​ ttps://twitter.com/majson/status/846325424468713473​ | and
@johanhaleby​ (creator of REST-assured) has commented, ​tweet1​ - ​tweet2​ - ​tweet3​ - and with that I’ve updated #37 to say “Partial” and with a link reporting
success in the wild called out.