Oracle Fusion Cloud - BIP Performance Tuning Tips and Documentation

During the early days of Oracle Cloud being adopted relative to SaaS technologies like ERP and HCM, it was quite common to develop extracts and reports using complex custom sql data models that would either be downloaded by users or scheduled to extract data and interface it to external systems. Overtime, Oracle has released guidelines and best practices to follow, and efforts like the SQL guardrails have emerged to prevent poor performing custom SQL from impacting environment performance and stability. To that end, I have been aggregating useful links to documentation around this topic from our interactions with Oracle Support over the past few months, which are consolidated in this post.

Links to Documentation:

• Optimize Performance of Multi-Thread Processes (oracle.com) [docs.oracle.com]This deals with multi-threading in HCM, this configuration will allow jobs to allocate more threads in the environment and therefore run faster.
• OTBI & BICC PSR Technote (2679006.1 [support.oracle.com]) and OTBI self-service [cloudcustomerconnect.oracle.com] monitoring
• The OTBI self-service dashboards can be downladed and installed in your environment so that you can see usage and monitor it. Some of the reports may not work after imported and could require modifications.
• BIP PSR Technote (2800118.1 [support.oracle.com]) and BIP self-service [oracle.com] monitoring
• Similarly to OTBI, there's BIP self-service reports that you can use to monitor performance and activity, and act upon those findings.
• BIP Datamodel validation enhanced with Saffron SQL pattern checks, available to customers in BIP Administration UI
• For custom BIP SQLs, Saffron [saffron.subnet1ad3phxsu.psr4saas01phx.oraclevcn.com] output along with the ref. to BIP Technote (2800118.1 [support.oracle.com]), that describes reported patterns in more details
• Fusion Applications BI Publisher : How to control BI Publisher Data Model time out (Doc ID 1921393.1 [mosemp.us.oracle.com])
• Fusion Applications BI Publisher : How To Configure And Use Audit In BI Publisher For Fusion SaaS Cloud Customers? (Doc ID 2059102.1 [mosemp.us.oracle.com]) to schedule monitoring of the health of your BI jobs.
• This is a key piece to proactive observing which Reports are nearing a critical threshold or identifying a sudden variation in runtime.
• Saffron report with tuning suggestions can be provided or you can use the new feature DM Validate (SAFfron results in Data Model UI) and fix the detected pattern by reviewing the Tech Note 2800118.1 [mosemp.us.oracle.com].
• For custom BIP SQLs, Saffron [saffron.subnet1ad3phxsu.psr4saas01phx.oraclevcn.com] output along with the ref. to BIP Technote (2800118.1 [support.oracle.com]), that describes reported patterns in more details
• Fusion Applications - BI Publisher Memory Guard Best Practices (Doc ID 2472630.1 [mosemp.us.oracle.com])

For scheduled reports, Oracle recommends the following guidelines:

• Having a temporary backlog (wait queue) is expected behavior, as long as the backlog get cleared over the next 24 hours.
• If the customer expect the jobs to get picked up immediately, submit via ‘online’ and wait – as long as they not hit 500 sec limit.
• If there are any jobs that need to be processed with high priority (over the rest), it's advised to mark reports as ‘critical’ so that they picked up by the first available thread.
• Oracle advises customers to tune their custom reports so that they complete faster and not hold threads for long time.
• Oracle advises customers schedule less impactful jobs during off-peak or weekend hours – manage scheduler resource smart.

Additionally, note the following:

• With Release 13 all configuration values including BI Publisher memory guard settings are preset based on your earlier Pod sizing request and cannot be changed.
• For memory guard the Oracle SaaS performance team has calculated and set the largest values that still provide a robust and stable reporting environment for all users to meet business requirements.
• The BI Service must support many concurrent users and these settings act as guard rails so an individual report cannot disrupt your entire service and impact the business.

Ultimately, effective instance management is critical for ensuring that your Cloud HCM system is running smoothly and effectively. Allocating resources based on the usage and demand will require co-ordination with various teams. There is a common misunderstanding that each HCM tool such as HDL, HCM extracts, or manual ESS job submissions operates on its own pool of threads. However, in reality, they all share the same ESS pool of threads. It is, therefore, advisable for customers to properly maintain and optimize their runbook to avoid overburdening the system and creating resource constraints.

Lastly, depending on the size of your pods, you have the option to allocate pods for specific tasks. For example:

• BulkLoading/ Performance testing/Payroll parallel runs: Pod with highest threads is a good candidate be utilized for bulk data loading, payroll parallel runs, and similar resource-intensive tasks such as performance testing.

The below graphic shows how ESS Threads are consumed, to exemplify the statements made prior:

Oracle Redwood Migration and Adoption

Oracle continues to migrate, re-design and implement new features utilizing their Redwood design system, and HCM continues to be a big focus relative to these efforts in the coming releases (it already has been, but it's really picking up steam now with the end in sight!). In yesterday's office hours for HCM Redwood adoption, hosted by Oracle, it was clear that Redwood will likely become fully mandatory by 25B, for HCM, and Oracle was clear that if you haven't opted in by then, that they would move you!

In this post we won't go into a lot of details relative to what Redwood is, but in short, it is a modern design approach, bringing an updated UI that leverages AI and ML very well, while delivering a more modern and engaging usability experience for the users. Instead, we will focus on key points relative to adoption that you will want to make note of.

We already covered that full adoption is planned for 25B (around the April 2025 timeframe), however, there's accelerated adoption points that need to be considered, meaning, you don't have until 25B to opt in for the below:

• Redwood Learning self-service mandatory for learners and managers (along with other select pages being enabled) - 24B
• Checklists and Onboarding replaced by Journeys (Redwood) - 24D
• Time and labor will transition to Redwood (24D), meaning features like timecards will have a new look and feel, etc.

The guidance right now is that current live customers start building an adoption roadmap and select lower environments to start enabling Redwood in desired areas to start performing impact analysis, and learning. There's also guidance for new implementations and in-flight implementations, and basically the message is to adopt Redwood to the maximum extent possible, to avoid significant changes soon after your Go-Live.

Oracle also urges clients to inventory their assets, and they list the below, as areas of consideration:

• Page Composer personalizations
• Transaction Design Studio personalizations
• AutoComplete rules: Defaulting and Validations
• Approvals and Notifications

To the above I would add your fast formulas (which it's not entirely clear whether significant impact is to be expected, and the question remains), and your visual builder extensions, particularly those built using the HCM embedded VB Studio capability. They also mentioned that security is the cause for 50% of reported issues with Redwood, because Redwood may require new security privileges, meaning, test your custom access and roles. Additionally, Oracle mentions that a tool will be released soon that can be used as a starting point to catalog assets and potential impacts, in the way of a report that you can run on your environment, the tool is called the "Redwood Adoption Analysis Tool". Make sure to monitor Cloud Customer connect for when that is announced, as it surely will come in very handy. On the topic of Cloud Customer connect, you can submit questions and request help there in the "Ask a Question" area, make sure to use the Redwood tag, and also, monitor the site for events such as webinars and office hours.

If you are a current live client (HCM), not utilizing Redwood yet to great extent, I recommend you dedicate an environment and go enable as much as you can there, then build a roadmap with milestones, and a test plan, etc. Additionally, change management will be very important, communications, trainings, job aid's, demos, because it is a significant change from a user experience standpoint.

Lastly, below are several useful resources relative to Redwood:

• MOS article on enabling redwood pages- HCM Redwood Pages with Profile Options – My Oracle Support Doc ID 2922407.1
• Fusion HCM: Redwood Required Steps for Environment Provisioned on Release 24A - Doc ID 2997123.1
• Extending Redwood Applications using Visual Builder Studio - Doc ID 2991662.1

Oracle Integration Cloud Gen 3 Upgrade - Tips and Features

As some of you may be aware, Oracle is migrating their OIC Platform to its new generation, called OIC 3. For those that remember the Gen 1 to Gen 2 migration, this one is more impactful, delivering a significant overhaul from a UI and usability perspective, as well as many new features. This entry will cover some of the features that I am most excited about, as well as important links and other key information relative to the upgrade effort.

An overview of key highlights with opportunities is below, and detailed documentation to all new features and other information is available at the bottom via links. Also, new updates are being rolled out to OIC Gen3 every month.

Gen3 Feature Highlights	Opportunities
Large payload support for integrations	Payload support is growing with Gen3, which allows for more complex use cases and larger volume to be processed, which will be a win already, but also can eventually lead you to rationalizing other tools away, such as MuleSoft and Biztalk, if you use those platforms.
Enhanced Disaster Recovery Capabilities	Gen3 will have an enhanced Oracle Managed DR offering that should simplify and reduce manual labor versus our existing DR strategy in Gen2, more on that below.
Private Endpoint Connectivity	This feature will set the stage to deprecate the connectivity agent used in OIC to connect to the DBCS or ADB (autonomous database) where you may house PLSQL for complex business logic, and instead allows us to connect OIC to the Autonomous Database directly, and if you have your own tenant ADB for VBCS, then both can coexist. In short, you can rationalize databases by removing the DBCS environments and consolidate VBCS and OIC into a single ATP (autonomous database), and while replacing the connectivity agent (third party) with an adapter (native), reducing technical debt and failure points, should you have this type of configuration in OIC 2 currently.
New recipes and adapters	Enhanced connectivity and opportunities both to accelerate development of new integrations and to reduce technical debt, such as custom integrations between OIC and custom identity or token providers.
Enhanced Event Framework	True pub sub capabilities will now exist in OIC, which will simplify when we use this pattern, as today we have to write parent/child integrations with custom hand offs, versus using native event capabilities internally in OIC. There also appear to be enhanced polling capabilities.
RBAC and Projects	This feature will enhance how you utilize global OIC environments across business units and teams, better allowing HR and AFT to co-exist on the same environment (for example), reducing risk and improving supportability. You can also move away from having independent Dev environments for HR and AFT or CX, and consolidate.
RPA Capability	RPA capabilities will be added to OIC, which will allow for the utilization of this feature to address RPA requirements and aid in removing dependencies with external tools and the utilization of less integrated and supported RPA capabilities, such as Blue Prism, in the context of the broader Oracle footprint.
Increased Observability	Changes to monitoring and API framework that will allow for enhanced monitoring capabilities.

To note:

• Not all features in Gen2 may yet be available in Gen3, and this upgrade also impacts VBCS and Process Automation, so extensive testing will be required, including performance testing.
• Oracle has mentioned that middle of next year will be the deadline to migrate, potentially August, and although initial research shows that many clients are still in Gen2 and that upgrade issues can happen, you should plan and execute this sooner than later.
• Some of our features weren’t ready or fully developed in OIC 3, such as the VBCS auto upgrade, and other features, and also that there’s several break/fixes that have been completed to stabilize OIC 3. As an example, Oracle was tracking an internal bug relative to stage files read/write, and this had a performance difference compared to Gen2, as an example 20-minute runtimes would go to an hour or two, which highlights A) the need to check the Known issues link, but B) to perform extensive regression testing, and also performance testing.
• I also recommend that you upgrade a single instance as a pilot, perform your test plans there, prior to moving to upgrading several environments at once.
• The VBCS and PCS assets within your Gen 2 instance will now be part of the auto upgrade process, which was previously not available, and would have caused significant manual work on your end, however, there's some outstanding licensing concerns Oracle is working through in this regard and some updates have been postponed until February, but if you have a simple instance without VBCS or PCS, there's no reason to wait, and you can go to OIC 3 already.
• Oracle utilized the auto upgrade process to internally upgrade many of their complex OIC assets in Q4 (which include VBCS and PCS), which has increased confidence in the process for those that will undertake the complex upgrade path.
• We confirmed that the IDCS stripe that the Gen2 instance is related to will not be impacted by the upgrade, and that OIC instance names, URL’s, etc. will also not be impacted.
• We confirmed the connectivity agent does have changes relative to how it authenticates (no longer supports basic authentication), but that it should be seamless during the upgrade process, but if you use a connectivity agent make sure to test it well.
• When you login to your Gen 2 environment, there will be a section that tells you if your environment meets the prerequisites to be upgraded, it is important these upgrade pre-checks pass, check the link below for more information. Also, if you don't want your environments to be auto upgraded if the pre-checks are met, it's important that you communicate with Oracle, to avoid surprises, you can raise an SR to postpone migrations.
• Lastly, our current understanding is that the deadline to migrate to OIC 3 will be August of 2024, currently, but this could move depending on adoption and issues.

Links to Documentation:

• What’s New in Gen3: Oracle Cloud What's New for Oracle Integration 3
• Differences between Gen2 and Gen3: https://docs.oracle.com/en/cloud/paas/application-integration/whats-new/index.html#GUID-6366FC51-5836-4D2E-AF8E-E9E939BF7330
• FAQ: https://docs.oracle.com/en/cloud/paas/application-integration/oracle-integration-oci/upgrade-faqs.html [docs.oracle.com]
• Upgrade pre-checks: https://docs.oracle.com/en/cloud/paas/application-integration/oracle-integration-oci/prepare-upgrade-oracle-integration-3.html [docs.oracle.com]
• Known issues: https://docs.oracle.com/en/cloud/paas/application-integration/known-issues/preface.html#GUID-34683717-9FA8-4982-BD27-39E1AAA43FBF
• Upgrade from OIC Gen2 to Oracle Integration 3 (OIC3) - Schedule Information (Doc ID 2977663.1)