{"_id":"581a0fed4883b30f00643ebc","project":"5615790c0f5ed00d00483dd1","__v":0,"category":{"_id":"5615790d0f5ed00d00483dd5","__v":19,"pages":["5615790e0f5ed00d00483dd7","561d48e46386060d00e06003","561d48fe31d9630d001eb5bd","561d49b657165b0d00aa5d8b","561d4a879463520d00cd11e2","561d67f48ca8b90d00210234","561d6a0bf0cff80d00ca22c3","561d6c5b071cd60d000d3221","562f9c2543c5570d001fe6bd","56311c99eae7ef0d00270e3d","56311d6702aff217007dba23","56311f96f1c0580d00fac719","563120b7242cda1900198b79","5631229bf1c0580d00fac721","563131559ead230d00a188f6","563134a324014b0d00bd9a4f","5631392082d96a0d00b0fb1d","56313c584b36120d00fdebfb","5642658ef424a10d00118360"],"project":"5615790c0f5ed00d00483dd1","version":"5615790d0f5ed00d00483dd4","sync":{"url":"","isSync":false},"reference":false,"createdAt":"2015-10-07T19:57:01.871Z","from_sync":false,"order":0,"slug":"opendns-investigate-rest-api","title":"Umbrella Investigate REST API"},"version":{"_id":"5615790d0f5ed00d00483dd4","__v":6,"project":"5615790c0f5ed00d00483dd1","createdAt":"2015-10-07T19:57:01.307Z","releaseDate":"2015-10-07T19:57:01.307Z","categories":["5615790d0f5ed00d00483dd5","56157b2af432910d0000f9fe","56157cfb0f5ed00d00483ddb","562684d95db46b1700fd4f48","573b7ea9ef164e2900a2b8ff","582e285d8373c20f00810608"],"is_deprecated":false,"is_hidden":false,"is_beta":false,"is_stable":true,"codename":"","version_clean":"1.0.0","version":"1.0"},"parentDoc":null,"user":"560b40145148ba0d009bd0b5","updates":[],"next":{"pages":[],"description":""},"createdAt":"2016-11-02T16:10:21.324Z","link_external":false,"link_url":"","githubsync":"","sync_unique":"","hidden":false,"api":{"settings":"","results":{"codes":[]},"auth":"required","params":[],"url":""},"isReference":false,"order":8,"body":"To perform a pattern search in the API, use the /search/ endpoint, append a RegEx pattern search to the API query and a start time.\n\nThe pattern search functionality in Investigate uses regular expressions (RegEx) to search against the Investigate database. There are several excellent tools online such as http://regexr.com to help if you’re not familiar with building RegEx. In the API, the default number of results returned is 100 but appending ?limit=XXXX parameter to any query can extend the results from 1 to 1000.\n\nIf the results of your query exceed these limits, we recommend refining the RegEx being used and performing multiple pattern searches.\n\nThe start parameter is required to begin your results. A typical use case for the start parameter is to run the same queries for strings of interest for the past couple of days. Alternately, you can provide a start parameter with a set time (in epoch time). An example of how to specify the start parameter in absolute or relative time is below.\n\nThe results extend back 30 days, and only discovers newly queried domains, whether that domain was registered recently or not. Both the UI and the API include a date which this domain was first seen, which is the first time we saw the domain being queried. This means that domains seen before that time period will not be not be included in the result, such as almost all common, well-known domains.\n\nThe results from the pattern search are generated from a database containing the information about domains that were looked up by Umbrella customers within the time periods specified. As such, you may find results that do not actually match an actual domain that resolves but that people were still doing DNS lookups for. We have done our best to sanitize these results, but domains that do not exist will occasionally appear.\n\n** As a note for using curl from the command line, many characters used in RegEx must be escaped or the command line interpreter will replace them. Thus it is necessary to use ‘\\\\\\*’ rather than just ‘*’, as well as for brackets (‘[]’) and parenthesis. **\n\nSample queries:\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"curl -H \\\"Authorization: Bearer %YourToken%\\\" \\\"https://investigate.api.umbrella.com/search/ope\\\\[a-z\\\\]dns.com?start=-1000minutes\\\"\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\nSample query with limit applied\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"curl -H \\\"Authorization: Bearer %YourToken%\\\" \\\"https://investigate.api.umbrella.com/search/ope\\\\[a-z\\\\]dns.com?start=-1000minutes\\\"\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\nSample query with start applied (absolute)\n\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"curl -H \\\"Authorization: Bearer %YourToken%\\\" \\\"https://investigate.api.umbrella.com/search/ope\\\\[a-z\\\\]dns.com?start=1428441014673”\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\nSample query with start applied (relative time)\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"curl -H \\\"Authorization: Bearer %YourToken%\\\" \\\"https://investigate.api.umbrella.com/search/ope\\\\[a-z\\\\]dns.com?start=-1days”\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\nSample query with categories included and relative start time\n[block:code]\n{\n  \"codes\": [\n    {\n      \"code\": \"curl -H \\\"Authorization: Bearer %YourToken%\\\" \\\"https://investigate.api.umbrella.com/search/ope\\\\[a-z\\\\]dns.com?start=-2weeks&includecategory=true”\",\n      \"language\": \"text\"\n    }\n  ]\n}\n[/block]\n### Parameter for input ###\n[block:parameters]\n{\n  \"data\": {\n    \"h-0\": \"Field\",\n    \"h-1\": \"Type\",\n    \"h-2\": \"Description\",\n    \"0-0\": \"expression\",\n    \"0-1\": \"string\",\n    \"0-2\": \"Required. A standard RegEx pattern search, must be URL-encoded.\",\n    \"1-0\": \"start\",\n    \"1-1\": \"string\",\n    \"1-2\": \"Required. Can either be specified in relative or absolute time. If specifying in absolute, use an epoch time (Unix time) millisecond timestamp within the last 30 days as the Start. If specifying in relative, use either seconds, minutes, hours, days or weeks with a minus sign in front. As an example -1days, -1000 minutes, or -2weeks are all valid. However these cannot be combined, you can only use one of the relative time enumerators per query.\",\n    \"2-0\": \"includeCategory\",\n    \"2-1\": \"boolean\",\n    \"2-2\": \"Optional. Default is false, if set to true this will include security categories in the results and may slow the return times.\"\n  },\n  \"cols\": 3,\n  \"rows\": 3\n}\n[/block]\n### Returned value for output if Success 200 ###\n[block:parameters]\n{\n  \"data\": {\n    \"h-0\": \"Field\",\n    \"h-1\": \"Type\",\n    \"h-2\": \"Description\",\n    \"0-0\": \"expression\",\n    \"0-1\": \"string\",\n    \"0-2\": \"This is the RegEx in the query as seen from the API. If results from your query do not match what you may have expected, check to see that the RegEx matches the one you tried to enter and that characters are correctly escaped in the query string.\",\n    \"1-0\": \"totalResults\",\n    \"1-1\": \"integer\",\n    \"1-2\": \"Total results from this search string. The default number of results is 100 and can be expanded using the limit parameter.\",\n    \"2-0\": \"moreDataAvailable\",\n    \"2-1\": \"boolean\",\n    \"2-2\": \"Whether more data is available than what is displayed. Will be true if totalResults exceed limit. We recommend refining your filter if this value is true.\",\n    \"3-0\": \"limit\",\n    \"3-1\": \"integer\",\n    \"3-2\": \"Default is 100, can be expanded to 1000 which is the maximum number of results for this endpoint\",\n    \"4-0\": \"matches\",\n    \"4-1\": \"array of strings\",\n    \"4-2\": \"Each match will contain the name of the domain matches, the and the first seen time, both in Epoch and ISO time format. Any matching Umbrella Security Categories will be here. This endpoint returns the security categories as strings rather than integers (eg: “malware”,”botnet”, etc) if includeCategory is true.\"\n  },\n  \"cols\": 3,\n  \"rows\": 5\n}\n[/block]\n\n[block:html]\n{\n  \"html\": \"<div class=\\\"api-code-block\\\">\\n  <div class=\\\"api-code-block__header\\\">\\n    <span class=\\\"api-code-block__header__label\\\">GET</span> \\\"https://investigate.api.umbrella.com/search/exa\\\\[a-z\\\\]ple.com?start=-1days&limit=10&includecategory=true\\\"\\n  </div>\\n  <div class=\\\"api-code-block__section\\\">\\n    <div class=\\\"api-code-block__section__header\\\">REQUEST</div>\\n    <pre>curl --include \\\\\\n     --header \\\"Authorization: Bearer %YourToken%\\\" \\\\\\n\\\"https://investigate.api.umbrella.com/search/exa\\\\[a-z\\\\]ple.com?start=-30days&limit=10&includecategory=true\\\"\\n    </pre>\\n  </div>\\n  <div class=\\\"api-code-block__section\\\">\\n    <div class=\\\"api-code-block__section__header\\\">RESPONSE <em>(HTTP 200, Content-Type: application/json)</em>\\n    </div>\\n    <pre>\\n{\\n  \\\"expression\\\": \\\"exa[a-z]ple.com\\\",\\n  \\\"totalResults\\\": 1,\\n  \\\"moreDataAvailable\\\": false,\\n  \\\"limit\\\": 1000,\\n  \\\"matches\\\": [\\n    {\\n      \\\"name\\\": \\\"example\\\",\\n      \\\"firstSeen\\\": 1432330927421,\\n      \\\"firstSeenISO\\\": \\\"2015-05-22T21:42:07.421Z\\\",\\n      \\\"securityCategories\\\": [\\n        \\\"Botnet\\\"\\n      ]\\n    }\\n  ]\\n}\\n    </pre>\\n  </div>\\n</div>\\n<style></style>\"\n}\n[/block]\n---\n[Domain Scores](https://docs.umbrella.com/developer/investigate-api/domain-scores-1/) < **Pattern Search** > [Co-Occurrences for a Domain](https://docs.umbrella.com/developer/investigate-api/co-occurrences-for-a-domain/)","excerpt":"","slug":"pattern-search-1","type":"basic","title":"Pattern Search"}
To perform a pattern search in the API, use the /search/ endpoint, append a RegEx pattern search to the API query and a start time. The pattern search functionality in Investigate uses regular expressions (RegEx) to search against the Investigate database. There are several excellent tools online such as http://regexr.com to help if you’re not familiar with building RegEx. In the API, the default number of results returned is 100 but appending ?limit=XXXX parameter to any query can extend the results from 1 to 1000. If the results of your query exceed these limits, we recommend refining the RegEx being used and performing multiple pattern searches. The start parameter is required to begin your results. A typical use case for the start parameter is to run the same queries for strings of interest for the past couple of days. Alternately, you can provide a start parameter with a set time (in epoch time). An example of how to specify the start parameter in absolute or relative time is below. The results extend back 30 days, and only discovers newly queried domains, whether that domain was registered recently or not. Both the UI and the API include a date which this domain was first seen, which is the first time we saw the domain being queried. This means that domains seen before that time period will not be not be included in the result, such as almost all common, well-known domains. The results from the pattern search are generated from a database containing the information about domains that were looked up by Umbrella customers within the time periods specified. As such, you may find results that do not actually match an actual domain that resolves but that people were still doing DNS lookups for. We have done our best to sanitize these results, but domains that do not exist will occasionally appear. ** As a note for using curl from the command line, many characters used in RegEx must be escaped or the command line interpreter will replace them. Thus it is necessary to use ‘\\\*’ rather than just ‘*’, as well as for brackets (‘[]’) and parenthesis. ** Sample queries: [block:code] { "codes": [ { "code": "curl -H \"Authorization: Bearer %YourToken%\" \"https://investigate.api.umbrella.com/search/ope\\[a-z\\]dns.com?start=-1000minutes\"", "language": "text" } ] } [/block] Sample query with limit applied [block:code] { "codes": [ { "code": "curl -H \"Authorization: Bearer %YourToken%\" \"https://investigate.api.umbrella.com/search/ope\\[a-z\\]dns.com?start=-1000minutes\"", "language": "text" } ] } [/block] Sample query with start applied (absolute) [block:code] { "codes": [ { "code": "curl -H \"Authorization: Bearer %YourToken%\" \"https://investigate.api.umbrella.com/search/ope\\[a-z\\]dns.com?start=1428441014673”", "language": "text" } ] } [/block] Sample query with start applied (relative time) [block:code] { "codes": [ { "code": "curl -H \"Authorization: Bearer %YourToken%\" \"https://investigate.api.umbrella.com/search/ope\\[a-z\\]dns.com?start=-1days”", "language": "text" } ] } [/block] Sample query with categories included and relative start time [block:code] { "codes": [ { "code": "curl -H \"Authorization: Bearer %YourToken%\" \"https://investigate.api.umbrella.com/search/ope\\[a-z\\]dns.com?start=-2weeks&includecategory=true”", "language": "text" } ] } [/block] ### Parameter for input ### [block:parameters] { "data": { "h-0": "Field", "h-1": "Type", "h-2": "Description", "0-0": "expression", "0-1": "string", "0-2": "Required. A standard RegEx pattern search, must be URL-encoded.", "1-0": "start", "1-1": "string", "1-2": "Required. Can either be specified in relative or absolute time. If specifying in absolute, use an epoch time (Unix time) millisecond timestamp within the last 30 days as the Start. If specifying in relative, use either seconds, minutes, hours, days or weeks with a minus sign in front. As an example -1days, -1000 minutes, or -2weeks are all valid. However these cannot be combined, you can only use one of the relative time enumerators per query.", "2-0": "includeCategory", "2-1": "boolean", "2-2": "Optional. Default is false, if set to true this will include security categories in the results and may slow the return times." }, "cols": 3, "rows": 3 } [/block] ### Returned value for output if Success 200 ### [block:parameters] { "data": { "h-0": "Field", "h-1": "Type", "h-2": "Description", "0-0": "expression", "0-1": "string", "0-2": "This is the RegEx in the query as seen from the API. If results from your query do not match what you may have expected, check to see that the RegEx matches the one you tried to enter and that characters are correctly escaped in the query string.", "1-0": "totalResults", "1-1": "integer", "1-2": "Total results from this search string. The default number of results is 100 and can be expanded using the limit parameter.", "2-0": "moreDataAvailable", "2-1": "boolean", "2-2": "Whether more data is available than what is displayed. Will be true if totalResults exceed limit. We recommend refining your filter if this value is true.", "3-0": "limit", "3-1": "integer", "3-2": "Default is 100, can be expanded to 1000 which is the maximum number of results for this endpoint", "4-0": "matches", "4-1": "array of strings", "4-2": "Each match will contain the name of the domain matches, the and the first seen time, both in Epoch and ISO time format. Any matching Umbrella Security Categories will be here. This endpoint returns the security categories as strings rather than integers (eg: “malware”,”botnet”, etc) if includeCategory is true." }, "cols": 3, "rows": 5 } [/block] [block:html] { "html": "<div class=\"api-code-block\">\n <div class=\"api-code-block__header\">\n <span class=\"api-code-block__header__label\">GET</span> \"https://investigate.api.umbrella.com/search/exa\\[a-z\\]ple.com?start=-1days&limit=10&includecategory=true\"\n </div>\n <div class=\"api-code-block__section\">\n <div class=\"api-code-block__section__header\">REQUEST</div>\n <pre>curl --include \\\n --header \"Authorization: Bearer %YourToken%\" \\\n\"https://investigate.api.umbrella.com/search/exa\\[a-z\\]ple.com?start=-30days&limit=10&includecategory=true\"\n </pre>\n </div>\n <div class=\"api-code-block__section\">\n <div class=\"api-code-block__section__header\">RESPONSE <em>(HTTP 200, Content-Type: application/json)</em>\n </div>\n <pre>\n{\n \"expression\": \"exa[a-z]ple.com\",\n \"totalResults\": 1,\n \"moreDataAvailable\": false,\n \"limit\": 1000,\n \"matches\": [\n {\n \"name\": \"example\",\n \"firstSeen\": 1432330927421,\n \"firstSeenISO\": \"2015-05-22T21:42:07.421Z\",\n \"securityCategories\": [\n \"Botnet\"\n ]\n }\n ]\n}\n </pre>\n </div>\n</div>\n<style></style>" } [/block] --- [Domain Scores](https://docs.umbrella.com/developer/investigate-api/domain-scores-1/) < **Pattern Search** > [Co-Occurrences for a Domain](https://docs.umbrella.com/developer/investigate-api/co-occurrences-for-a-domain/)