YAML format of the Code Gaps and Multichoice questions

There is an option on our platform which allows you to create code gaps and multichoice questions using YAML format. Please find more detailed information below:

yaml1.JPG

Code Gaps Questions

Before you begin - what is an Code gaps Question anyway?

Code gaps is a multipurpose task. You can upload a snippet of code with gaps in it where the candidate must fill in the blanks to make it complete.

code_gaps.png

=== What should I do?

Your task is to prepare the Code gaps questions based on your experience in the particular technology. Make sure to check the existing tasks on the platform not to have the duplicate questions. If you need any help with that - let us know :)

== How Code Gaps Questions are created

It is possible to create code gaps questions on the platform. Creating them for the Devskiller platform is really easy! A Code gaps question can be added manually on the platform using the web interface or can be imported from a YAML file.

If you want help with the right format here is our https://gitlab.com/devskiller-tasks/documentation/documentation/-/blob/master/code-gaps-questions/yaml_task-schema.json[yaml_scheme] that you can import to your editor.

=== YAML file format

A YAML file should consists of a list of questions. Every question is a hash, which must contain following keys.

* `title` - title of a task. Title structure - Technology | Framework/Library | Unique task name which describes tested skill.
* `difficulty` - the difficulty level, on of `EASY`, `MEDIUM`, `HARD`.
* `skills` - look at the TalentBoost skill map and write out the whole path to the skill. Ask your PM for more information.
* `tags` - skill tags, ie. what kind of knowledge this questions tests. It should be the technology name and topic if you have one.
* `question` - the actual question which will be presented to the candidate.
* `type` - type of a task.
* `mode` - syntax highlight mode.
* `content` - the task code. The potential gap should be covered with `{{{and}}}`.

Please see a full example of a question below.

.code-gaps-devops.yaml
[source,yaml]
----
---
- title: DevOps | Running docker containers - cleanup
difficulty: EASY
tags: [DevOps, Docker]
skills: [Infrastructure, Containers, Runtime, Docker]
question: |-
Ensure that the container will be removed when it exits
type: CODE_GAPS
mode: PLAIN
content: |-
$ docker run {{{--rm}}} hello-world
----

.code-gaps-sql.yaml
[source,yaml]
----
---
- title: JavaScript | Some unique task name
difficulty: EASY
tags: [JavaScript]
skills: [Software Development, JavaScript]
question: |-
Fill in the JavaScript code gap to make the code do this and that.
type: CODE_GAPS
mode: JAVASCRIPT
content: |-
here goes the code and here goes the {{{|C|gap|C|gaps}}}
----

===== List of possible highlighting modes:


APL, PGP, ASN.1, Asterisk, C, C++, Cobol, C#, Clojure, ClojureScript, Closure Stylesheets (GSS), CMake, CoffeeScript, Common Lisp, Cypher, Cython, Crystal, CSS, CQL, D, Dart, diff, Django, Dockerfile, DTD, Dylan, EBNF, ECL, edn, Eiffel, Elm, Embedded Javascript, Embedded Ruby, Erlang, Esper, Factor, FCL, Forth, Fortran, F#, Gas, Gherkin, GitHub Flavored Markdown, Go, Groovy, HAML, Haskell, Haskell (Literate), Haxe, HXML, ASP.NET, HTML, HTTP, IDL, Pug, Java, Java Server Pages, JavaScript, JSON, JSON-LD, JSX, Jinja2, Julia, Kotlin, LESS, LiveScript, Lua, Markdown, mIRC, MariaDB SQL, Mathematica, Modelica, MUMPS, MS SQL, mbox, MySQL, Nginx, NSIS, NTriples, Objective-C, OCaml, Octave, Oz, Pascal, PEG.js, Perl, PHP, Pig, Plain Text, PLSQL, PowerShell, Properties files, ProtoBuf, Python, Puppet, Q, R, reStructuredText, RPM Changes, RPM Spec, Ruby, Rust, SAS, Sass, Scala, Scheme, SCSS, Shell, Sieve, Slim, Smalltalk, Smarty, Solr, SML, Soy, SPARQL, Spreadsheet, SQL, SQLite, Squirrel, Stylus, Swift, sTeX, LaTeX, SystemVerilog, Tcl, Textile, TiddlyWiki, Tiki wiki, TOML, Tornado, troff, TTCN, TTCN_CFG, Turtle, TypeScript, TypeScript-JSX, Twig, Web IDL, VB.NET, VBScript, Velocity, Verilog, VHDL, Vue.js Component, XML, XQuery, Yacas, YAML, Z80, mscgen, xu, msgenny.

=== General rules

When you are creating Code gaps questions please follow the following set of rules:

. Group all related Code gaps Questions in one file.
. The file should be a correct YAML file, named with `.yaml` extension.
. Markdown can be used for formatting including the syntax for code blocks.
.. For the inline code snippets use backticks (```) like in markdown.
.. For the block of code use 3 backticks (`````````).
. Start each sentence from the capital letter.
. End each sentence with a dot.
. Use 'Fill in the gaps…to…' in the question to indicate what exactly should be done in this task. For language-specific tasks also mention technology name in the question to show candidate which language they have to use to complete the code.
. Each task can have more than one gap. Please, make sure to have more gaps for Hard tasks.
. Do not mark a full line of code as a gap. It should be short.
. Remember to add mode as a code highlight such as Java or SQL.
. Make sure to enable check spelling in your editor! :)
. Whenever quotation marks and brackets are involved consider multiple possible answers eg with ' ' also ” ” could be correct; with tomek() also tomek( ) or tomek( ) could be correct.
. Validate your YAML file before starting a Merge Request - you can use https://github.com/adrienverge/yamllint[yamllint] or an http://yaml-online-parser.appspot.com/[online parser].

=== Best Practises for Code Gaps and Multi Choice Questions

. Avoid asking “What will this code print?”
.. Questions like that are too easy to simply paste into the editor and see the right answer. Preventing copy-paste is just a partial solution that developers can easily bypass.

Good: Why this code doesn’t print “Foo”
Bad: What will this code print?

. Avoid asking things that can be googled literally in seconds
.. See if pasting the your questions into Google reveals the answer immediately on StackOverflow

Good: How does hashCode() function influence HashMap performance?
Bad: What is the pessimistic performance of HashMap

. Avoid asking things that can be looked up in the documentation
.. Using Google is fine, so don’t verify knowledge that can be looked-up easily in reference documentation.

Good: What’s the purpose of DataInputStream?
Bad: Does DataInputStream have a readDouble() method?

. Avoid asking direct questions, focus on problem solving
.. Do not ask direct statements that can be easily memorized. Instead ask problems that can be solved with that particular knowledge.

Good: If you want to allow subclasses only to access certain methods, which modifier would you choose?
Bad: Which access modifiers exist in Java?

Good: Which inheritance strategy would you choose to improve performance of base class queries in Hibernate?
Bad: What are the different inheritance strategies in Hibernate?


=== How to mark a Code Gap

* The potential gap should be covered with `{{{and}}}`.
* If more than 2 choices can be correct, put || - in front of each correct answer.
* You can also combine rules for code gaps. For example: {{{|CW|"hello"|CW|'hello'}}} - This means that the correct answer can have `'` or `"` and ignore case + ignore whitespace included.
** `|C|` - stands for _ignore case_ in the answer.
** `|R|` - stands for _regexp_ in the answer.
** `|W|` - stands for _ignore whitespace_ in the answer. If the answer consists only from 1 word no `|W|` is needed.

*NOTE*: If your task shoul have `{` and `}` as a part of the proper code, you need to use the following format:

```
{{{ {this is all code gap} }}}
```

=== Notes on line continuation in YAML

There are multiple ways to achieve a line continuation in YAML.

. http://www.yaml.org/spec/1.2/spec.html#id2796251[Folded style] - removes the newlines within the string (but adds one at the end).
[source,yaml]
----
title: >
DevOps | Running docker containers - cleanup
----
. http://www.yaml.org/spec/1.2/spec.html#id2795688[Literal style] - turns newlines within the string into literal newlines, and adds one at the
end.
[source,yaml]
----
title: |
DevOps | Running docker containers - cleanup
----

You can use either of them in your YAML files. Please check https://stackoverflow.com/questions/3790454/in-yaml-how-do-i-break-a-string-over-multiple-lines[this thread] for more information.

Multiple Choice Question

Before you begin - what is an Multiple Choice Question anyway?

Choice Questions are essentially a quiz question. They can be used, in addition to programming tasks, to test candidate knowledge on a particular subject. Our platform supports Single and Multiple Choice Questions but in general we refer to them as MCQs (Multiple Choice Questions).

multichoice.png

=== What should I do?

Your task is to prepare the MCQs based on your experience in the particular technology. Make sure to check the existing tasks on the platform not to have the duplicate questions. If you need any help with that - let us know :)

== How MCQs are created

It is possible to create multiple choice questions or single choice questions on the platform. Creating them for the Devskiller platform is really easy! An MCQ can be added manually on the platform using the web interface or can be imported from a YAML file.

If you want help with the right format here is our https://gitlab.com/devskiller-tasks/documentation/documentation/-/blob/master/multiple-choice-questions/yaml_task-schema.json[yaml_scheme] that you can import to your editor.

=== Correct and wrong answers

You can create two types of questions: multiple and single choice questions.

It is pretty obvious that in single-choice questions there could be a lot of possible answers but one correct, but in multiple-choice questions, there can be any combination of correct vs wrong answers. For example: 2 correct and 5 wrong; 4 correct and 1 wrong and etc.

=== YAML file format

A YAML file should consists of a list of questions. Every question is a hash, which must contain following keys:

* `title` - title of a task. Title structure - Technology | Framework/Library | Unique task name which describes tested skill.
* `difficulty` - the difficulty level, on of `EASY`, `MEDIUM`, `HARD`.
* `tags` - skill tags, ie. what kind of knowledge this questions tests. It should be the technology name and topic if you have one.
* `skills` - look at the TalentBoost skill map and write out the whole path to the skill. Ask your PM for more information.
* `question` - the actual question which will be presented to the candidate.
* `type` - type of a task >> MULTI_CHOICE.
* `duration` - change task duration in minutes. There is a default time set per task difficulty: EASY = 1 min; MEDIUM = 2 min; HARD = 3 min.
* `choices` - list of hashes with choices. The key in these hashes can be either `correct`, for correct answers, or `wrong`, for wrong ones.
* `mode` - add `SINGLE` type of mode if there is only one correct answer. If your task has more correct answers, simply ommit this key.

Please see a full example of a question below.

.multiple-choice-javascript.yaml
[source,yaml]
----
---

- title: JavaScript | Demo question
difficulty: EASY
tags: [JavaScript, SecondTag]
skills: [Software Development, JavaScript]
question: |-
Select the correct statement about JavaScript:
type: MULTI_CHOICE
duration: 3
mode: SINGLE
choices:
- correct: JavaScript is a dynamic programming language, which can be used to write client-side scripts for web browsers.
- wrong: JavaScript applications are compiled to bytecode that can run on a Java Virtual Machine.
- wrong: JavaScript was originally developed by James Gosling at Sun Microsystems
----

=== General rules

When you are creating MCQs please follow the following set of rules.

. Group all related MCQs in one file.
. The file should be a correct YAML file, named with `.yaml` extension.
. Markdown can be used for formatting including the syntax for code blocks.
.. For the inline code snippets use backticks (```) like in markdown.
.. For the block of code use 3 backticks (`````````).
. Start each sentence from the capital letter.
. End each sentence with a dot.
. Correct answers should be randomly ordered.
. Use 'Select all that apply' in the question with more than one correct answer. Remember that candidate gets score only when all correct answers were selected.
. Make sure to enable check spelling in your editor! :)
. Validate your YAML file before starting a Merge Request - you can use https://github.com/adrienverge/yamllint[yamllint] or an http://yaml-online-parser.appspot.com/[online parser].
. Add a proper tag! Always and keep attention to that!
. Task name format: Technology | Library, framework | distinct title. For example: JavaScript | React | DOM manipulation.
. Make sure not to repeat the question which is already available on the platform!

!IMPORTANT: Please make sure to follow these rules strictly or the task will not be accepted

=== Best Practises for Multi Choice Questions

. Avoid asking “What will this code print?”
.. Questions like that are too easy to simply paste into the editor and see the right answer. Preventing copy-paste is just a partial solution that developers can easily bypass.

Good: Why this code doesn’t print “Foo”
Bad: What will this code print?

. Avoid asking things that can be googled literally in seconds
.. See if pasting the your questions into Google reveals the answer immediately on StackOverflow

Good: How does hashCode() function influence HashMap performance?
Bad: What is the pessimistic performance of HashMap

. Avoid asking things that can be looked up in the documentation
.. Using Google is fine, so don’t verify knowledge that can be looked-up easily in reference documentation.

Good: What’s the purpose of DataInputStream?
Bad: Does DataInputStream have a readDouble() method?

. Avoid asking direct questions, focus on problem solving
.. Do not ask direct statements that can be easily memorized. Instead ask problems that can be solved with that particular knowledge.

Good: If you want to allow subclasses only to access certain methods, which modifier would you choose?
Bad: Which access modifiers exist in Java?

Good: Which inheritance strategy would you choose to improve performance of base class queries in Hibernate?
Bad: What are the different inheritance strategies in Hibernate?

. Avoid asking questions that are based on subjective opinion
.. If you think that such question might be useful, please make sure to ask these questions using the Essay task type

Bad: Which JavaScript Framework is better Vue, React or Angular?

=== Notes on line continuation in YAML

There are multiple ways to achieve a line continuation in YAML.

. http://www.yaml.org/spec/1.2/spec.html#id2796251[Folded style] - removes the newlines within the string (but adds one at the end).
[source,yaml]
----
title: >
JavaScript DEMO question
----
. http://www.yaml.org/spec/1.2/spec.html#id2795688[Literal style] - turns newlines within the string into literal newlines, and adds one at the
end.
[source,yaml]
----
title: |
JavaScript DEMO question
----

You can use either of them in your YAML files. Please check https://stackoverflow.com/questions/3790454/in-yaml-how-do-i-break-a-string-over-multiple-lines[this thread] for more information.

Was this article helpful?
0 out of 0 found this helpful