ITB 2023 - cbElasticSearch Modern Searching for Modern CFML - Jon Clausen.pdf

With over 59,000 ForgeBox installations, cbElasticsearch is the most widely-used, fluent, and robust way for CFML developers to leverage the power of Elasticsearch. In this session, we take a deep dive in too the ways you can leverage Elasticsearch in your search, in your UI, ingesting historical files to make them searchable, and creating robust metrics, charts and visualizations of your data. We will also explore the new time-series and index lifecycle functionality in cbElasticsearch - which can be leveraged for centralized logging and auditing.

1. CBELASTICSEARCH
MODERN SEARCHING FOR MODERN CFML

2. Me: Jon Clausen
Senior Solutions
Architect,
Ortus Solutions
Grand Rapids, Michigan

3. SEARCH
EXPECTATIONS:
AN EVOLUTION
•As latency in response times
has decreased ( e.g.
broadband speeds ), the
expectations for search
responsiveness has
decreased
•A decade ago, visitors were
willing to wait for their
results. Not anymore. Users
expect pages to load in 2
seconds or less

4. SEARCH
EXPECTATIONS:
AN EVOLUTION
•Users expect a variety of
filters to meet their specific
search requirement
•Users expect “fuzziness” in
their search terms
•Mobile devices continue to
be game changers and
make search relevancy and
response more critical than
ever

5. SEARCH
EXPECTATIONS:
AN EVOLUTION
•Abandonment increases
when:
•Search response latency
increases
•Relevancy decreases
•Slow response times are a
double whammy - with
your visitors and for SEO

6. CFML SEARCH
<cfsearch
collection = "collection name"
name = "search name"
category = "category[,category2,...]"
categoryTree = "tree location"
contextBytes = "number of bytes"
contextHighlightBegin = "HTML string"
contextHighlightEnd = "HTML string"
contextPassages = "number of passages"
criteria = "search expression"
maxRows = "number"
orderBy = "rank_order"
previousCriteria = "criteria"
startRow = "row number"
status = ""
suggestions = "suggestion option"
type = "criteria">

7. CFML SEARCH
•Functionally and
programmatically robust
•Reasonable response times
•Limited in query complexity
and conditionals
•limited to no aggregation
capabilities
•Generally requires
additional data fetching to
display result data
<cfsearch
collection = "collection name"
name = "search name"
category = "category[,category2,...]"
categoryTree = "tree location"
contextBytes = "number of bytes"
contextHighlightBegin = "HTML string"
contextHighlightEnd = "HTML string"
contextPassages = "number of passages"
criteria = "search expression"
maxRows = "number"
orderBy = "rank_order"
previousCriteria = "criteria"
startRow = "row number"
status = ""
suggestions = "suggestion option"
type = "criteria">

8. ELASTICSEARCH

9. WHAT IS
ELASTICSEARCH?
•Open-source, RESTful,
distributed search and
analytics engine built on
Apache Lucene
•Quickly evolved become
the most popular web
application search engine
•Used for log analytics, full-
text search, and
operational intelligence

10. WHAT IS
ELASTICSEARCH?
•Free, open source software
which can be run on-
premises or using a variety
of cloud-based providers
•Interacts with applications
through a REST API, using
a JSON-based query DSL

11. ELASTICSEARCH
BENEFITS
•Fast - 10s on SQL vs 10ms
on Elasticsearch
•Intuitive APIs
•Fast Index Updating
•Schema-free JSON
document storage
•Performative on very large
datasets

12. ELASTICSEARCH GLOSSARY
•index - like a table in a relational database
•term - an exact value match in elastic search
•text - ordinary, unstructured text
•keyword - non-fuzzy, exact match field
•field - a key/value storage pair in a document
•DSL - Domain Specific Language

13. THE ELASTICSEARCH DOCUMENT
{
"numberOfVersions": "1",
"documentationURL": "",
"isActive": true,
"createdDate": "2014-08-23T00:25:41+00:00",
"downloads": 0,
"typeSlug": "commandbox-modules",
"entryID": "0660E39A-1F51-4287-8375DCCF220D619F",
"entryType": {
"typeName": "CommandBox Modules",
"typeSlug": "commandbox-modules"
},
"isListed": true,
"description": "<div>This is a CommandBox command called \"norris\" that will display
Chuck Norris facts.</div><div><br></div><div>If you don't have CommandBox, you'd better
get it before you get roundhouse-kicked in the teeth. &nbsp;Start here:</div><div>http://
www.ortussolutions.com/products/commandbox</div><div><br></div>",
"isPrivate": false,
"numberOfRatings": "0",
"bugsURL": "",
"typeName": "CommandBox Modules",
"collaboratorIds": [],
"hits": 3638,
"slug": "CommandBox-Chuck-Norris",
"sourceURL": "https://github.com/bdw429s/CommandBox-Chuck-Norris",
"homeURL": "https://github.com/bdw429s/CommandBox-Chuck-Norris",
"updatedDate": "2020-11-02T21:20:10+00:00",
"latestVersion": {
"version": "2.0.0.0",
"entryVersionID": "bedd0826-2f2b-11e6-8e43-001c429c2760",
"createdDate": "2014-08-23T00:25:41+00:00",
"updatedDate": "2016-06-08T08:46:34+00:00"
},
"avgRating": "0.0000",
"installs": 3995,
"title": "CommandBox Chuck Norris",
"summary": "This is a CommandBox command called ",
"organization": {}
}

14. THE ELASTICSEARCH QUERY DSL
{
"query": {
"bool": {
"must": [
{
"range": {
“latestVersion.createdDate": {
"gte": "2014-08-23T00:25:41+00:00"
}
}
},
{
"term": {
“entryType.typeSlug": “command box-modules”
}
}
]
}
}
}

15. CBELASTICSEARCH
A COLDBOX MODULE WITH A FLUENT API FOR CREATING, RETRIEVING,
ANALYZING, AND AGGREGATING ELASTICSEARCH DOCUMENTS
box install cbelasticsearch

https://cbelasticsearch.ortusbooks.com

16. THE CBELASTICSEARCH QUERY
var search = getInstance( "SearchBuilder@cbelasticsearch" ).new( “);
return search.term( "vendorCode", "CB" )
.dateMatch(
name = "startDate",
start = dateFormat(
now(),
"yyyy-mm-ddT00:00:00+00:00"
)
)
.execute();

17. THE CBELASTICSEARCH QUERY:
DOCUMENT SEARCH
var searchResult = getInstance( "SearchBuilder@cbElasticsearch" )
.new( "forgebox-entries" )
.term(
"entryType.typeSlug",
"commandbox-modules"
)
.dateMatch(
name = "startDate",
start = "2018-06-01T00:00:00+00:00",
end = "2018-06-30T23:59:59+00:00"
).execute();
searchResult.getHits().each( function( hit ){
... do stuff with hit struct ...
} );

18. THE CBELASTICSEARCH QUERY:
AGGREGATIONS
return getInstance( "SearchBuilder@cbelasticsearch" )
.aggregation(
"forgebox-entries",
{
"terms" : {
"field" : "entryType.typeSlug",
"size" : 20000,
"order" : { "_term" : "asc" }
},
"aggs" : {
"slugs" : {
"terms" : {
"field" : "slug",
"size" : 20000
}
}
}
}
)
.execute();

19. THE CBELASTICSEARCH QUERY:
AUTOSUGGEST
// return an array of auto-suggest results with scores
return getInstance( "SearchBuilder@cbelasticsearch" )
.new( "forgebox-entries" )
.suggestPhrase(
"Chuck Norris",
"mySuggestions",
"title"
)
.execute()
.getSuggestions()[ "mySuggestions" ].options;
// example option struct
{
"text": "CommandBox Chuck Norris",
"score": 0.8,
"freq": 1
}

20. PIPELINES
getInstance( "Pipeline@cbelasticsearch" ).new( {
"id" : "foo-pipeline",
"description" : "A test pipeline",
"version" : 1,
"processors" : [
{
"set" : {
"if" : "ctx.foo == null",
"field" : "foo",
"value" : "bar"
}
}
]
} ).save();

21. LOGBOX APPENDERS
appenders = {
console = {
class="coldbox.system.logging.appenders.ConsoleAppender"
},
logstash = {
class="cbelasticsearch.models.logging.LogstashAppender",
// The log level to use for this appender
levelMax = "ERROR",
// Appender configuration
properties = {
// The name of the application which will be transmitted
"applicationName" : "My Application name",
// A release version to use for your logs
"releaseVersion" : "1.0.0",
// The max shard size before rollover
"rolloverSize" : "1gb"
}
}
}

22. SHOW AND TELL

23. GETTING STARTED:
box install cbelasticsearch

https://cbelasticsearch.ortusbooks.com

24. Q & A

25. CBELASTICSEARCH
MODERN SEARCHING FOR MODERN CFML