coreBOS OpenSource comes to an end. After 9 years of an incredible journey, the time has come to change direction.

If I look back, I can see the first version, the conception of coreBOS in 1998 (or so, far away and a little blurry now). That first code already had the idea of making things customizable and adaptable to create the perfect software for any company. At that time, I was alone (hmmm recurring word in my life), the internet wasn't what it is today and open source was still an emerging buzzword. So it took me 5-6 years to land in the vtiger CRM universe and find an application that I could take where my coreBOS vision was. A lot of effort, a LOT of effort and sacrifice later, I had made myself comfortable with that code base, but the inexistent community didn't want to take the product where my vision was. I tried, really hard, but the people there consistently ignored me to the point where I decided to fork vtiger CRM, apply all the magic we were already giving our clients, and start coreBOS.

Fast forward to today, I, and the people who believed in this project, have created a software platform that is very close to the vision I had. A really powerful, customizable, and adaptable framework that can help most companies.

The curious fact about that is that we have started to see adoption by software houses, which makes sense as that is what the software is for, but we fear the same situation Elastic Search had with Amazon. That threat and the difficulty, or direct fallacy of an open source business model that could sustain the project economically has taken us to continue the development under a much more strict license and close the software to give us legal power.

The new license is closed and proprietary. The code is now owned by a company and any rights to resell or distribute it must be negotiated with that company. Further development effort will be closed and accessible only to partners and collaborators. I have negotiated a special license agreement for existing community members and standalone users who have supported the project up to today, and the new company is founded on the idea of reverting all the effort invested in the next years back into the open source world after some years. We are making a bet to see how it turns out. Let's see what happens

You can reach out to me for the full details of the license, or if you have any questions.

The imminent future isn't going to change much, I'm still the project manager of the software and the development team is the same (and growing), the current community members are shielded so they can stick around as much as they consider necessary, maybe even could be good for them. For example, now we can enforce exclusivity for them,... who knows?

Farther into...

Read about the impact of updating the Smarty library to 4.3.0.

In the past days, I have updated the coreBOS Smarty template library to the latest version (4.3.0). They have introduced a breaking change: PHP functions can no longer be used directly as modifiers.

This means that we can no longer use any PHP function after the tube (|) operator. We can only use the native Smarty supported modifiers.

So, from now on,

{$SOME_VARIABLE|vtlib_purify}

must be

{vtlib_purify($SOME_VARIABLE)}

This means that as of today:

• I will not accept any pull request that has the unsupported syntax (even if I haven't officially pushed the update yet). We must all start using the new syntax immediately.
• We have to update all the deprecated calls in coreBOS (read below)
• We have to update all the deprecated calls in our custom modules

I didn't get into the details but it seems that both for the language compiler and for some philosophical stance they want to distinguish supported modifiers from any PHP/coreBOS function. So we can use their internal modifiers, like escape, replace, cat, (a whole bunch of others), ...

{$header|replace:' ':''}

is valid while

{'LBL_Hide'|getTranslatedString:'Settings'}

is NOT because getTranslatedString is a function external to Smarty even though there is no visual indicator of the fact. Whatever ...

I analyzed all the templates in coreBOS and found that 80% of the incorrect calls are for getTranslatedString, that is all over the place. The other 20% was a mix of different PHP and coreBOS functions (a lot of vtlib_purify). I migrated all the 20%. For getTranslatedString I added it to Smarty as a plugin (see below). So we can still use getTranslatedString as before because coreBOS adds it to Smarty as an internal modifier. This is done with one line of code and it makes it indistinguishable from the native modifiers. Really, I don't get it, anyway...

So, with the change I have made, all of coreBOS base code will work again as normal. Usual stuff, a lot of work no one sees to obtain the same result.

The issue is with our custom modules. Any custom module that has its own templates MAY be using a PHP function as a modifier and will need to be migrated to the new syntax. This is important information: we must update the custom module templates as we find these errors To see the errors, work with error_reporting and display_errors on. The deprecation message will appear on the screen. It must be fixed and the corresponding repository updated.

I suppose mostly you will run into the vtlib_purify function, that is the one that gave me most work. I also had some html_entity_decode and urlencode functions.

To add the getTranslatedString support I had to add this line of code:

diff --git a/Smarty_setup.php b/Smarty_setup.php
index 1c06bb084..2770186c7 100644
--- a/Smarty_setup.php
+++ b/Smarty_setup.php
@@ -51,6 +51,7 @@ class vtigerCRM_Smarty extends Smarty {...

Twelve new global variables, a bunch of PHP 8.2 preparations, Wizard View, System Notifications, GenDoc, Workflow enhancements, Master Detail, Web Service new credentials access,... Wow!

• Business Actions
  • support for new fields like width, height, and others when creating from code
  • add type for MasterDetail Button
  • dynamic labels by rendering field values in context: this permits us to use field names in the label like, Change Status to $somestatusfield
  • sort types picklist to make them easier to find
  • support error and success messages in Launch Workflow Button business action
• Calendar
  • add link type to notification task moreaction column
  • fill in contact when the related entity is selected
  • pending tasks widget: card and table output
• GenDoc:
  • permit not adding template name to saved document
  • support draw shapes at top-level document
  • permit setting the name of the generated document using expression maps. you can use the context variables for expression evaluation: GDTemplateID, GDTemplateName, GDTemplateNumber. Use the GenDoc_Attachment_Name global variable.
  • support for UnoServer conversion. deprecating unoconv: GenDoc_Convert_URL_UnoServer
  • format date fields directly and support dots in date marker format: you can now format any date field following its name with a colon and the desired format. You can even use dots for the formatting.

GenDoc Test Dates Open Office Document

• Master Detail functionality
  • add support for Business Actions
  • hide/show master detail in detail view from map
  • support inline edit for uiypes 19 and 21
• Documents
  • adding modal preview
  • get folders document belongs to method: getFolders()
• System Notifications
  • return notification ID when adding notifications
  • truncate subject text
  • show only related records in the pending Tasks widget
  • support RECORD substitution
  • function to change status and convert fields to JSON so the implementor can easily set ID to change
• Wizard wow!! great enhancement. Thanks Denald and Xhilda. We have to get some blog posts and videos about this!!
  • add FilterFromContext parameter
  • filter records in wizard related with the current record
  • filter steps from context values
  • init for FORM templates
  • open wizard everywhere in the app
  • select ids after clear filter
  • start setup forms in wizard
  • start supporting expressions in wizard
• getFieldValuesFromSearch return values based on user preferences
• add Load Filter option to Potentials related list in campaigns
• easily add buttons on the dependent list: powerful enhancement to the related list where we can add buttons through code
• add aftersend action to PHPMailer e-mail sending: cbEventHandler::do_action('corebos.email.aftersend', $mail)
• support URL elements in top level menu
• add code to support button to execute workflow in Mobile UI
• set callback function for ProcessInfo
• writeReportToCSVVariable parameter to get Reports as a response instead of output to screen
• support Application_ListView_Default_Sorting in TUIGrid
• implementing dropdown action buttons for TUIGrid
• merge uitype 2 into 1: they are the same, 2 is useless
• isTime support for seconds in Validations
• Web Service
  • authentication token-based access using the Credentials module and corebos-authorization header.
• Workflow: get a lot of attention this month!!
  • support expression inside strings Fantastic time saver enhancement!! Thank you Malik
  • add Notification workflow task: set Notification ID in the context
  • char expression...

Another "how-to" post. Learn how to add some action buttons in coreBOS.

First, we will define a button on the detail view of a record that will trigger a workflow and refresh the view.

We create a workflow that does some action, in this case, I'm going to use an already existing workflow on the potentials module. Potentials have a workflow that updates the forecast amount on every save. This workflow in my test installation is 12. So we can add a link with this business action:

As you can see it calls an existing function runBAWorkflow and appears as a link. We can change that to a button using DETAILVIEWBUTTON (at the top) or DETAILVIEWACTIONBUTTON (in the action panel) instead

And, in this case, we can also use the Launch Workflow detail view widget. The widget gives us more control over the final output.

This works, but the action does not refresh the data. To do that we can do two things. Looking at the code we see that there is a hook that we can work with. You can read about how this works here. For our case, we have to add another business action with some JavaScript to listen on the hook and launch the dtlViewReload function to refresh the data in the browser.

<script>
function corebosjshook_runBAWorkflowBefore(workflowid, crmids) {
  dtlViewReload('Potentials', crmids);
}
corebosjshook.before(window, 'corebosjshook_runBAWorkflow', corebosjshook_runBAWorkflowBefore);
</script>

This code connects to the hook and executes a data reload when the workflow has finished.

The nice thing about this is that you can add any code and logic you want, but if you just want to reload the data the runBAWorkflow function has an additional parameter that does that. We just have to add true in the fourth position or refreshdv=true in the launch workflow widget like this

module=Documents&action=DocumentsAjax&file=LaunchWorkflowButton&workflow_id=12&record=$RECORD$&type=slds-button slds-button_outline-brand&button_label=RUNWF12&refreshdv=1

Now let's see how to do this in the list view. In The list view, we have a different goal, it is a mass action button that will affect all the selected records instead of just one as we did on the detail view. As before, coreBOS gives us a function to do this called runBAWorkflowFromListView so our business action looks like this

You can see that we also have the refresh list view parameter (set to true in the example above) and the hook which we can also use to call the refresh function listViewReload().

So the same logic.

There is one more option in the list view which is to execute the command as a server-side event script and output the results in the same window the mass edit functionality uses. This gives...

Enhancements everywhere and a lot of new widgets (very happy about this), code cleanup, and an important performance optimization. Impressed, as usual.

• Pending Tasks widget and Notification Panel enhancements

• DETAILVIEWACTIONBUTTON business action type permits us to add buttons instead of links in the detail view action panel.

• Document PDF Preview
  • simplify query and support loading any document with docid parameter
  • support for business map
  • add view PDF link to document detail view
  • add documentation link
• GenDoc conditional blocks
• List View collapsable actions

• adda a new icon to the Mobile detail view to return to the list view

• support for custom action with CANCELGO parameter in Cancel button
• getConditionsFromSQL2CV convert SQL to custom view filter syntax
• getFieldValuesFromRecordRecursively ability to add multiple setFields
• refactor getEntityFieldNames to create getEntityFieldNamesByTableName
• add an Open menu option to Kanban that takes you to the detail view of the record

• disable database logging in key-value store
• relate Messages on Leads conversion
• set default queue manager to database
• move the filter creator to hover in Key Metrics table
• support ProcessInfo popup as validation and business action
• ProcessSetting functionality action
• convert RECORD to CRMID if given in screen_values in Validations
• in the Database abstraction layer: use the logging method everywhere and add a switch to deactivate logging
• add Slider component

• support linked columns in Table, Grid, and Pie question types. coreBOS will use the grouping column to autogenerate links for those questions that have a group by option
  • get the correct filter conditions for the linked tables option
  • set conditions on empty values of grouped link table entries
  • correctly set list view conditions for reference fields (uitype10)
  • use the correct property to get the field name
• Web Service
  • update or create Products if upsert_conditions is set in the map
  • add meta infomation for fields in related modules
• Workflow
  • getRequest workflow expression function
  • support multiple IDs for getFieldsOf expression

• coreBOS Standard Formatting: eliminate warnings, eliminate useless code, variables, and comments. Format code. General, Accounts, Duplicate, Import, Install, Business Question, JSLibraries, modules, RecycleBin, Settings, Templates, Utilities, Vtiger
• Documentation:
  • function headers, and comments
  • Launch Work Flow widget example of call
  • non-stop wiki enhancements
• Security
  • Images: eliminate unneeded grouping from security check regular expression, optimize and correct them
• Optimizations
  • delete obsolete migration files
  • denormalize CRMENTITYREL. This is an important speed optimization. The generic many-to-many relation was based on one database table that can relate two entities in two ways, so the code had to use an "OR" condition to get the correct related records. With this change, we now have a database table that we can query with one condition. The difference is very important.
  • Custom View: eliminate unused variables, move variable calculation
  • List View: delete an unused variable and function call, use empty instead of isset
  • Tracker: move variables
  • Calendar: move action construction to function, move action Smarty assignment to separate function, move pending task query and card output to functions
  • PopupFilter: moving object instantiation into condition scope, using...

Read about a surprising comparison with PDFMaker and learn about some new enhancements in our document and PDF construction and generation system: GenDoc.

This year the first Sunday of the month is also the first day of the year, so I start by wishing you all a great, happy and productive year ahead.

Last month I implemented a couple of GenDoc templates while migrating from PDFMaker and I was surprised at how easy one of the requests was compared with the solution implemented in PDFMaker. Note that I have very little knowledge of PDFMaker so I don't know if the solution I found implemented there can be done in some other easier way using that tool, I am just comparing the solution that I found there with the way I solved it in GenDoc and how much more flexible and easy it is.

So, the request is to list all the time control records related to an invoice in a table with this output:

The solution in PDFMaker is a 350 line PHP custom function full of HTML and CSS. When I saw that I was a little depressed as I thought that we could not do that in GenDoc. In fact, we can't. We cannot create a piece of code that generates a whole section to insert into a generated document. I mean, obviously, we can, as we are doing more complex things manipulating the open office document internally, but it is not an easy task, it isn't that anyone can just add some function and we include the output in the generated open office document.

So, I stepped back and decided to approach the problem from a coreBOS point of view. Given the desired output, how to implement that with what we have, not the way they do it in PDFMaker? Six minutes later I had this open office template.

Wow! That was easy.

The sum cells are missing and can't be done in the GenDoc template, those calculations have to be done in coreBOS. So I added two custom fields and an update field workflow task that looks like this.

aggregate_time('Timecontrol', 'totaltime')

The final GenDoc template looks like this:

Looking at the solution and comparing it with the PDFMaker custom function I see that the flexibility of GenDoc is incredible. The template is easy to read and modify, with no CSS, no HTML, no PHP, just a nice WYSIWYG interface. All the calculations and conditions to aggregate are done inside coreBOS using the standard workflows expression language, so all the customizations are in the hands of the implementors, not the programmer, and it took me 30min. I can't imagine how much time it took to create the 350 line custom function.

Since we are talking about GenDoc I would like to share with you a new functionality that we added recently: conditional sections.

For some time now we have been asked to be able to include or not a section of...

Customization month. New widgets, workflow enhancements, global variables, business actions, and business maps to make the application even more flexible. We also get some very interesting features in the application itself. coreBOS just keeps going!

• Menu typing access. Nice!

• viewtype support for global variables. Now we have another escalation option in global variables, we can set variables depending on the view we are in. So we can have two different values for a variable in EditView and DetailView. This was necessary for some of the new variables that were created this month. Thanks Malik
• Help notification panel. We started a project to add context-sensitive help in coreBOS. This is merely the first step where we add the panel and make it available in workflows as can be seen in the next screen. Constantly growing step by step.

• Business Maps
  • Field Dependency now permits to apply mode to actions at a global action level, not just individual actions, and also supports multiple view modes separated by comma
  • PopupFilter BusinessMap that permits us to define the action a uitype 10 field should have when opening its popup. Thanks Malik
  • PopupFilter add formodule and relatedmodule parameters in conditions for supporting filters in the popup
  • process dynamic variables from the front end
  • AdvancedSearch BusinessMap for saving presets. An interesting map that permits us to define a set of conditions that we want to be predefined when we open the advanced search. The use case was for a user that always has 5 search fields and had to add them each time. Now we define them in the map and the fields are preselected. Thanks Malik
• initial support for widgets on top detail view using the top:// directive

• Documents PDF preview Widget. Thanks Timothy

• add Slider widget using a Vanilla Js carousel library Thanks Denald and Xhilda
• the Table, Grid, and Pie formats Business Question now supports clickable fields/sections for grouped questions Thanks Denald
• add BusinessActionInformation to the widget context
• Global Variable now always returns the related business map if it is selected. This breaks backward compatibility.
• we can now mass update file fields from the ListView. Thanks Denald
• Mobile list view can now access both edit and detail view directly. Thanks Malik
• Mobile: correctly set URL when the protocol is given
• Edit view now support three special parameters which are Business Maps: FILTERFIELDSMAP, FILTERDEPMAP, and FILTERVALMAP. These variables override (or complement) the default maps. This was added to support special fields, dependencies, and validation in the new popup edit functionality.
• application modifications to support Process Settings business pattern
• add type of question field to Survey Question
• TiddlyWiki: use title for entityname only if entityname is not already given
• get the master related id in the MasterDetail popup edit view
• support for custom action with CANCELGO parameter in the Cancel button. Developers can now define what this button should do.
• support for unique HTML ID on application message DIV, which opens the possibility to have more than...

Some business processes require asking for more information during the steps. This information can be different from step to step or simply be some settings that need to be related to the record. This post will show you how to configure your coreBOS to do just that.

BPM more information popup enhancements

We have added a "more information" popup screen configuration module to coreBOS. Initially it was linked to the BPM Process Flow Perspective but then we modified it so it could be used independently of the BPM. In both options it is a very interesting business pattern.

You can read the details in the repositories:

• BPM Process Flow Perspective
• BPM Process Flow More Information Settings
• BPM Process Flow More Information modules

And see how they work watching these videos.

Configuration Settings per record business pattern

Another business pattern we have solved recently is when you have some more or less advanced configurartion options that must be applied per record in a module. Instead of saving the settings in different unrelated fields inside the module itself, this new Process Settings module makes it easier and more user-friendly.

You can read the details and learn more about in these links:

Module Process Settings

Making coreBOS more and more configurable!

Photo by Giulia May on Unsplash

A lot of things to celebrate. Denald not only made more commits than me but has scaled to the second position in the contributors' list. An incredible feat. Congratulations! We have a new developer and community member who has started strong. Welcome Malik. The list of enhancements is big this month and it is very distributed among our contributors which makes me very happy. All in all a great month!

• Global Search: a VERY useful enhancement to make the global search much more powerful. Thanks to Malik, we can now use:
  • #:: to search in tags, equivalent to tag::
  • $:: to search in currency fields, equivalent to currency::
  • site:: to search in URL fields, equivalent to url::
  • @:: to search in email fields, equivalent to email::
  • - to exclude words from your search: Put - in front of a word you want to leave out. For example, jaguar speed -car
  • quotes to perform a search with an exact match: Put a word or phrase inside quotes. For example, "tallest building"
  • .. to search within a range of numbers: Put .. between two numbers. For example, camera $::50..$::100
  • OR to combine searches: Put OR between each search query. For example, marathon OR race
  • you can combine OR and an exact match
• recover support for related module fields in Advanced Search
• add autocomplete feature to uitypes 11, 13 and 7. Thanks Kiko
• Business Actions
  • add widget size and header fields: we can now indicate if we want to have a header or not and the width and height of the widget. Thanks Timothy
  • add alert fields: for those actions that return a success/failure result we can indicate the message to show directly in the record. Thanks Denald
  • show a custom alert message in Actions
• Business Map
  • change RelatedListBlock map format to accept multiple modules
  • show on another module the tree related list block, tooltip
  • Field Dependency: add support for MODE based conditions at the dependency and field level. This permits us to define dependencies that will be applied or not depending on the state of the action. We support:
    • mode = 0: dependency is applied in all modes
    • mode = 1: dependency is applied when creating a record
    • mode = 2: dependency is applied when editing a record
    • mode = 3: dependency is applied when viewing (DetailView) a record
  • Field Dependency: new directive to include dependency or actions from other records. We can now load a whole branch of the XML into a dependency or an action making these maps easier to maintain and reducing the amount of duplicate code. This looks like this:

    <dependency>
    <loadfrom>map name or id</loadfrom>
    </dependency>

• convert SQL checkbox to a picklist with Query Type in Business Question. The first step to supporting Clickhouse in the questions. We also merge some pending Clickhouse enhancements. Thanks Timothy
• Widgets:
  • LaunchWorkflow action button widget. Can launch any workflow from a button with styling. Thanks Timothy
• Delete Handler to reset Remaining Units when Check Invoice Lines. Thanks...

Hacktoberfest 2022 and coreBOS.

This year, as the previous ones, I tried to create the four Pull Requests on projects different than coreBOS. So I started out the first weekend of October searching in GitHub and flipping through hundreds of open source projects to see which ones I could help. I found some interesting projects which I ended up adding to my tiddlywiki for future reference and did six pull requests. Three of them were in supported Hacktoberfest repositories and three were in repositories that were not participating but I had some interest in.

I dedicated about 30 minutes to Mautic documentation. I had in mind to create one or more pull requests in that project but it turned out that I had to learn how to work with Read the Docs and I saw that the new documentation project was still in an early phase so it would take me more time than I wanted to dedicate. Note that I am totally aligned with the work of Read/Write the Docs, I believe in what they are doing, I just couldn't do that now.

While reading the many interesting open source projects out there, none really catched my attention enough to dedicate significant time to, so I started thinking that I should pickup one or more of those coreBOS tasks that we have in our endless to-do list, which, I know, we will never get to and do those. As I read the different projects and implemented the six pull requests, this idea got stronger until I landed on the Hi-Folks/statistics project.

HiFolks/statistics is a PHP package that provides functions for calculating mathematical statistics of numeric data. It is like the library PHP needs to get closer to Python. I liked it as soon as I saw it. My mind related it to the workflow system where we could add it to make many calculations with the data saved inside coreBOS. I put it in the back of my mind and continued scanning. Shortly after, with three pull requests already accepted, I decided to dedicate the rest of my time for Hacktoberfest to include the statistics library into the workflow system as my fourth pull request.

Some days later, coreBOS had support for a rich set of statistical calculations on data in the workflow system. Since many parts of the application support the workflow engine, we can use those calculations in all of those places. I recorded a video presentation of the functionality.

I want to thank Digital Ocean, and all the supporters and participants in Hacktoberfest for this awesome event.

By the way, by the time I finished implementing the coreBOS enhancement one of the three pull requests in the non-participating projects was accepted, so the statistics enhancement was not officially my fourth pull request, but I still count it as a hacktoberfest effort.

See you again next year!