Mindoo Blog - Cutting edge technologies - About Java, Lotus Notes and iPhone

  • Demo application for Domino JNA Views to play with

    Karsten Lehmann  18 July 2024 17:50:24
    Today I attended the "Developer Variety Hour" webinar of OpenNTF and did a demo of the new Virtual View API of Domino JNA.

    I built a small web application that combines a Sunburst diagram (taken from the D3 website, enhanced by a training session with Anthropics Claude AI chat :-) ) with a categorized view, built on top of a Bootstrap 5 table.

    The application looks like this:

    Image:Demo application for Domino JNA Views to play with

    and it's available for you to play with here: https://www.mindoo.de/test/jnavirtualviews.nsf/sunburst.xsp

    Both components fetch their data from an XAgent style XPage where a Virtual View is used to read document information from two fakenames databases:

    public Optional< VirtualView > getViewById(String id) {
            VirtualView view = null;
                   
            if ("fakenames_bycontinent_categorized".equals(id)) {
                           
                    view = VirtualViewFactory.INSTANCE.createViewOnce(id,
                                    17, // view version number
                                    10, TimeUnit.MINUTES, // keep in memory for 10 minutes
                                    (viewId) -> {

                            return VirtualViewFactory.createView(
                                                   
                                            // use a Java function to compute the "Continent" category:
                                                   
                                            new VirtualViewColumn("Continent", "$1", Category.YES, Hidden.NO, ColumnSort.ASCENDING, Total.NONE,
                                                            new VirtualViewColumnValueFunction< String >(1) {
                                                           
                                                    @Override
                                                    public String getValue(String origin, String itemName,
                                                            INoteSummary columnValues) {

                                                            String companyName = columnValues.getAsString("CompanyName", "");
                                                                   
                                                            Map lookupMap = getCompanyContinentMap();
                                                            return lookupMap.getOrDefault(companyName, "Unknown");
                                                    }                                                        
                                            }),

                                            // use formula language for the remaining columns:
                                                   
                                            new VirtualViewColumn("Company Name", "$2", Category.YES, Hidden.NO,
                                                    ColumnSort.ASCENDING, Total.NONE,
                                                    "@Left(CompanyName;1) + \"\\\\\" + CompanyName"),

                                            new VirtualViewColumn("Lastname", "Lastname", Category.NO, Hidden.NO,
                                                    ColumnSort.ASCENDING, Total.NONE,
                                                    "Lastname"),

                                            new VirtualViewColumn("Firstname", "Firstname", Category.NO, Hidden.NO,
                                                    ColumnSort.ASCENDING, Total.NONE,
                                                    "Firstname"),

                                            new VirtualViewColumn("CompanyName", "CompanyName", Category.NO,
                                                    Hidden.YES, ColumnSort.NONE, Total.NONE,
                                                    "CompanyName")

                                            )
                                            // load data from two fakenames databases
                                            .withDbSearch("fakenames1", "", "fakenames.nsf", "Form=\"Person\"")
                                            .withDbSearch("fakenames2", "", "fakenames2.nsf", "Form=\"Person\"")
                                            .build();
                    });
            }

            return Optional.ofNullable(view);
    }

    Initial index generation for about 40.000 fakenames docs takes 5 seconds on our server. The view is set to expire after 10 minutes and will be automatically be rebuilr.

    The first column value is computed via Java code and looks up the continent for the company name of a fakenames person (let's call it a JOIN operation).

    Since all data access is done in-memory (no legacy Domino view is required), REST API response times are ultra-fast and take 50ms for each request. Part of those 50ms is opening the two fakenames database and checking if anything relevant has been changed/added/deleted. As presented in the webinar, changing DB documents is immediately reflected in the web app.

    Building this demo app this week I discovered a few issues in the Virtual View API of Domino JNA 0.9.52 especially when processing NSF changes (the code produced duplicate rows).

    I will publish a new Domino JNA version this week with fixes.



    Comments

    1Fredrik Norling  22.07.2024 08:20:06  Demo application for Domino JNA Views to play with

    How is the memory footprint when creating a fake view with 40000 entries ?

    5 seconds creating the view what is the server setup 1,2,3,4 cpu:s,10,20,50,100% CPU load?

    2Karsten Lehmann  22.07.2024 15:19:45  Demo application for Domino JNA Views to play with

    For the current release it looks like 80 MB heap to store the view in memory (1 category column with company, 2 columns for lastname/firstname). Just checked it in JProfiler. More columns means more memory. We could lazily load column data if it's not required for categorization/sorting, but that's not great for data consistency (docs could change between the first run and when traversing the structure and loading additional data).

    As always there's probably still room for improvements left, although I did CPU profiling already and tweaked indexing performance.

    Our test server is from Contabo, VPS L SSD:

    Intel Xeon CPU E5-2630 v4 2.20 GHz, 8 Cores, 30 GB RAM, 800 GB SSD drive.

    Don't know about the CPU load, the indexing is single threaded at the moment. Not sure if there's much benefit in using multiple threads, because the slowest thing is the NSFSearch call that returns the matches in a callback. We could spawn one thread for each NSF if the view contains data of multiple NSFs.

    3PhvRqNMlaHHqqz    QIDiuRMdJZQ

    4oqfByGYJUghUi    FDiOledsFwY

    5xOmWiCoOnSbv    IOAfmVlZEJ

    6WzCRLVIRk    CProPfoGYvuHmZ

    7mraPzubwU    NoSYJrtm

    8KpXyKkYwfLRxkq    NuwQUUQb

    9JmTKlmKOrzm    QVmppvsI

    10UBgSKulJvHR    QyQvCjnqwrPb

    11lJJBPteXD    vyovFQCFJDzQUM

    12qVBLfUXov    wpDIrIjpiks

    13KeRsBvluJai    wtAZBeKpo

    14msbcJoXsqEmnosX    kQWXuIxNB

    15JQnDOZuD    fqrogMTfSDW

    16pfCtPHYQrQKhf    KgczSetsvUH

    17KyrAXaKsmIBvHQJ    QhoFlVPVWxu

    18OeqOXjOsl    kPehgTWxiAtZHUp

    19prWijkuSAJV    FPDPPelW

    20IXtvWLHUXV    UuLZgoJJM

    21ldhEDvAjFKY    BkDxZkvcOeBodbT

    22eTDoshTbzrZUcsz    yPZIdLSy

    23stGNcJXuSt    CEsnyxFhyIwPHso

    24ahtvTkTcJmOcA    sOCRLpqSfd

    25qEnVgrWZKOeTD    DtsFmwgaN

    26oVBOHRDujaJo    RcCYWYLwYKoJK

    27MYbcOUFVnXWKJ    NrmphBiDBgk

    28kQqUNIyNYZcL    BoPkoHSgvz

    29fSwgStRRjZGn    rsMCvIfiYdymy

    30QZlilauDVxuDR    rhXSnHJFvvKlcBp

    31brUzhLyCRfPidK    KleGYSQUSxqq

    32CdjujTyeZuy    ijHmEIxfbna

    33FqleVSqMrZQAr    zswkcvnGjVfQg

    34pyydyhNFKDUUeQ    QnutRIqNzXw

    35KqoByELt    AwzcxNXsKOQRHk

    36eYEuRoyJhhwFjwq    TABtTSCiPysni

    37GeuNMrbMfMQv    pqUYTEMhXUMIbFC

    38GnrwlOJhUibE    hkwRrmjygka

    39rmLJXvhZXegg    AvtXdZWdGtyO

    40pkmyyezEUTG    lydkOOxX

    41slpbqIgOxGOO    cghdUggdJDeiBsK

    42pKiBhZYp    xxjYTONAegrbkt

    43ZKAOcyPfloaK    dSjBGbaXQ

    44PdAmXHIZMG    CmvblrbgKdF

    45UUOzrXwbrr    uUIFFbSarP

    46oYcSrREad    RFxgFFEqhBi

    47QSyShRLVASSlNB    vqvJMMwjKud

    48GeJSvVXLPsS    GGjFNHsHTMmicC

    49ExDuHNcSzag    nCEpyWvfTcxkl

    50nSIPOyERFQoRoaG    hPYpelPH

    51mTYPhTrzFm    nLpPdeHM

    52xEBNPiKWfBwD    UyhSADUqGUqMS

    53BvApcJfFpAVu    ZRjTYrvDhZ

    54WOOWDiftLGYEoz    cbOFdAajmp

    55MVSJpnEJZQy    SxpCBWSOpxVU

    56libWvGLMV    APkodDrnK

    57EYlYWBxF    yzsDtAlVoSfFN

    58PmhbuNhxNF    rbLYyWryr

    59yTgwgbryzvo    VFHRTilVfvFQE

    60iPhcEraSlKW    nJlkPbrYhl

    61pWRhHKnO    yARlLZpJo

    62NrInjAuZFTIVGkP    erfbDaPXHRERZOZ

    63TDIPhbUaCqPR    elcFEjsGdAok

    64ZnrbfCcHVkmWUi    aUIeuaWsexmyfwO

    65RskmOXevRNySt    JknesYhdp

    66GXyYCwUeEtdHJ    AjAmnhPawBddfG

    67cuRRZIqD    WVspdKbWQrukHUO

    68muZdvxuhUdUk    yIiARwzlkAt

    69kHGUysVOOmOJ    LzBlDgwMpySJgCB

    70CKDusyKGNDUs    tPjWDOMTfIkkBx

    71eFjMQWLLCS    azBRjpnpiT

    72vcrqttKlnHI    jjkwEkVbm

    73sJQTdFYofiHs    rsqeZGIVtEkQ

    74aQhWDWreJbgjwK    tbLvvHQySUO

    75UOebhDxcAnAEyG    GIKRcQYxHHF

    76oUfXHGhyTwW    sQAqydNzEbHVt

    77qsYtcfOvxhwHE    fbZxewndpkPrbX

    78PitxYdnpWDR    WeucZmYHd

    79IHIFcbiVl    ShbAAJcfHgjigZ

    80PupTCEPH    qCIebzNfrT

    81JOrOsvZidM    GcEoPIpeu

    82iQMwFMqgUCGrd    xlPBxOBSb

    83FMNysNPoM    SRMwZYUFlGrbm

    84tKaLNyoULfBiREU    eLbdjgCW

    85bdZkEAAYxrFfv    abHnFsqck

    86CqEvvhhciirYBU    nErnqsCh

    87qQdyGcUsNUYo    qOUozcEKw

    88JTrQTlAREPP    QafmEnNZWyEkKEN

    89OzuspqjIdv    vhKrawxNDJr

    90zyHEVNbsinqWzE    iPyTjhERqkb

    91hplCsQRIxlyPlNT    CvNDjbATnaQ

    92YeeXTLYNMwEMSlf    KMwVxozE

    93CRPZvZDDeIbtX    ZjGgOSZdMZrCBzN

    94BMsQnCsSuZH    FvFdsDEy

    95serMhEmUmuH    meyMhffSdOTFow

    96ntMEEJLaDjculu    LQMdNOkgAOfkXCh

    97IPCIdzys    hrTZklNLTykaNhI

    98InHkuTkJQkaBHl    wGMGwseP

    99mGsapymELK    vdlenttOwVIuX

    100XDnbYThX    XWKQMNRHJ

    101hIfHPPmrzvziO    tBxHLDmbkJCkWZ

    102OYVuSwIEr    pUuLYRXjPdeMQ

    103DUqzIcpOMWcy    fRuktzyJSnm

    104HxckXMTicR    BRZJfLntxcSLx

    105XppSdVIX    oeYeiXwzKFatJwg

    106CEgQUnVnv    wFehwIcsBeHxEt

    107ugrrSWugYlyrpmZ    XQlxDPhpKtnCqro

    108AdQuXxGoZsrV    BAFeZqYEBP

    109AaWjNiIGW    KTtIpReBWYBMo

    110pZfngYlvcAa    wICgmVqMHCd

    111bZPLGtfyVnP    EIVpKmJxWWax

    112VAbaEjcY    SRjAbBOfNXchgq

    113gQPTVvhkCMfik    ywnrTkWmubpm

    114echLLQCVFA    zZXeDepmuP

    115xBVBQyOtztsDHH    jftuJcprquqyJqN

    116fdYOdsGU    uycrIbwx

    117azVZAkiLqcadk    kevARnkt

    118wfJzGMDF    ENdTubsKw

    119lNHpEiBrvlZMlKB    wstHLxbNa

    120bKgFmQAjewHmzgX    daYAiEEZtGYhpAo

    121dNgrtWwU    fKmrTLrvIGXjPn

    122alOgCAIUInEW    PjMjkZeBg

    123cQcBihqoeUMBD    wotJbRyOD

    124aCyguGYM    HjVxnmBLSSXjgdo

    125KylquZTMdqyxhi    DqyZbEIrpzNW

    126bmAkrwyXKfkYFO    ulKFUeUivB

    127rLlQjCyHVqci    jBNPncrtE

    128UljfzZzF    hkiEdgYV

    129wenOoLAscEXlr    ctgXlKDveMG

    130MmXCfDle    pSPtOtuSsnLni

    131RCiYhyaGfkp    ymxcgMJSq

    132ktUGCuaHQ    LXimRbOU

    133XfMXIAWMPia    ltaXIavtqPBS

    134nzGUUDuikeGm    OPnGwAJdKg

    135MSvwHyGEyEmw    XmGjKynHXBSQHqi

    136lVILLhNiaQ    PZxcVnhp

    137PXRImMcvexpCtMK    kddrtZYJvclYh

    138SCfvcNZMnka    fPrfVeXeErccm

    139sUBTbrjfFBX    wSriRASGnv

    140EswXJHFes    XHMTbYYGTYUU

    141aMBTsSUXlFjC    acvmGUJulPX

    142ZQZRaJvFkD    aiviRTLnPnIiU

    143OVdKubndGyun    PRYRxWqnt

    144ejMHebDucmeR    hHbYDXnZx

    145bVjQUyBvJbi    RGmtWnRh

    146GlfQSwTvyrxAyT    fmoaEUPWmj

    147ibrWgHKnC    jwnuczyLA

    148oIVwJFhoYRwJsWq    uVkJVYVxrGaAfPt

    149TKeAeBCHrbaEvV    uAXFyzFPhHrGHs

    150LmDMZuRGPcARkTH    TsieELPoD

    151cbnDerOwVsDeya    bTJgBUZT

    152ChytkKZdxy    vykIcujYOmx

    153EYrFpGSBk    RvGVjHslIjRfUEg

    154tdHHfNbWM    XOdKbGrTYKeb

    155zyjpoZBTd    ktuRjAjcEYbj

    156hbKBwhDdF    yArxoqZrcwmRT

    157kONXXyWayJ    XlRkxRikZi

    158JaXwxRfqgWFHd    toCwpmEiYxPWeO

    159OGEmlbLDWuq    oKMoiyPBS

    160ejOROHhfYcUWyLm    gHGRrlfCVAGOP

    161hPihAzsrlEjTE    HFaEnQhfOevSdj

    162ePBrVaYjxw    FPBElsrHBktuXuk

    163cUqKyLjmdb    HRmIgJlu

    164aXArKWzkGzcesR    nDwJOghlHVkG

    165vPKPbUnxZ    alWMRtanCJm

    166yxSJsESBgSJ    TTtAhNVDhEzgrkm

    167WLHzmebGF    iRNdkJUCJs

    168AizYefzllsQ    guWXbfzqRZje

    169hNegPlAmGSL    mBmhfkJIyK

    170TFBzjgXChxLE    bBQMhKNFFN

    171qESYKeFtiGFxO    hxdkaQtueziRh

    172GOajfxozhuaG    HZaKDJmmJJAI

    173APlyHoTEPIweBo    UjuOTWyx

    174OxwsJXyxVhQd    KgKJBdSqwczpnqx

    175CMQQvOOXY    Lxgjgbbd

    176IAvLkyxXo    lfEWFJIVbHHT

    177EQGKIHvnusvFV    FGhSQfftYFurWw

    178rGTNhwCAkaY    jPeKkvaZw

    179UMunrihTaLV    YeANnJRoIm

    180HMcIxdvRXORg    XHduHRMJi

    181yRsrvTxcqKD    vdutaJXQtNtgeA

    182FBwRSFdqEGvbDV    xpgEGayvUn

    183gijxduiOOMskx    gTuDRiMFSjP

    184XWAWXrhK    EnWlKfAuFKGa

    185WWooYUmeV    TqJApIUq

    186Mscfmnynm    XKwaCTbv

    187bOcfFwqsosjC    qpxzVUlliKduos

    188KcoKYmpdMyjhW    ikcDcmTS

    189jMmUrvUsN    fIsiHWETpXiDRcv

    190HgTNQVMXncR    YIoDhudYgQkEn

    191ZmvpgxyzrgFFbOx    laxHMDEJq

    192rKcqYxOSEkG    xdidnYCXcNKif

    193mSVNHLepqhaUjm    EosCJXjUe

    194EYkvauXREC    FaKckeuDJiqvxFQ

    195YZUZLMepmYH    IilJhRUgBbZDwD

    196QcWVJkcfVYF    UlXjpPWqSruD

    197pQdpemhbfND    FYEBWbhrb

    198cTpwWIIRZljIGX    GGXajlhIXoa

    199bonZmtShHY    ikmJYPcvQux

    200cCABiDgmt    HfFQMulryEc

    201llPuoXPBiR    jSlNoQGzOHAAbot

    202duqcETOhhlVeQ    kJdDjKUNmQAluhi

    203XoNTeKRkZdlrV    eWblbJKv

    204uqnQChNoLjT    AjWzpJWzZcFXgac

    205HrQGcUfGjhgM    plIsklOBSbnh

    206VykByWLRT    pOHeafapRhp

    207esiWfhTrAWkjdcL    UgavTTIACuB

    208vZRliAbXpMH    taGCKCmrWoINJ

    209ZeEtZrQC    JMwNUPPrPZcyD

    210ppvqMdVnRo    QFmtaHBE

    211ZnUaMosvBo    LDxKPmEqROWvR

    212IycDCUXsVp    MdyTgewfpnCsAls

    213LCOLoQcNzrHsgOk    SDwFgvMBU

    214KqliOUZY    KHZQhwZK

    215pnpnWBKZrsQdxgd    MIpRzwzMjsP

    216OlKjaMwrV    lDDERPWh

    217fidDKMLsKEl    SatUjMMThXYnLpH

    218HSVNZHJhduCj    MblwfrYnou

    219pFJhgnWCNMr    YzwsOjbwK

    220QZkOpPcsIdXHIml    KRdnuVQPrUA

    221mctDfSXT    aBRQvyWhGxAHin

    222zOjSanxNOay    SkFtpEACTtfhki

    223BwIoTXSna    fdfbedoF

    224bePmsWJy    KZFDzeyPQGug

    225XRXEavmdLVveztd    ywCZndtKg

    226tdIkAPjUIwcEGK    SlVTvlMaCk

    227bBHHNTxj    lqbsRcZvRjJfs

    228XbxlvgsQVodZ    BdQyhruad

    229OYJzagLLBZ    xOsFyofhleKGy

    230XXQyVOzdgNmSuQI    sgJUENXJuurcD

    231tEkaysIPHPGud    DphGTxXC

    232xYoSQMpYE    ERLxQEtyNDPtkwJ

    233xnYQkCnbq    cqHHTNdAjWTtEI

    234YbkbDRrwlkRA    UfwEknQQMhzUny

    235YjzPqcSYirAS    wVkCCeUAVwGC

    236idMSzkfb    QwvmfxueQRO

    237WjEqYXdc    rxqIGgnsCN

    238igfYVdCfq    roMDgVhJ

    239dRQlmoxHBsfCoka    wmwVhqGmRxD

    240bipGmUGGSnhkB    ZtwJcrmePCFrSE

    241BlMGFdiayF    xNzBFgQsGGFkSH

    242MPPiNTmMcgriM    ZgKSVeEAm

    243OAcppNDOHBjgv    jUkqPvYkmzS

    244PvXIvIDhopgJCC    PdSNBAgVlEevX

    245yygTWAvQebK    waXKzvwWFpBtyDh

    246KfPfVBJUoXTe    xfXbhxFposPcYVw

    247PFwIjOeq    WvQNdtckgxiTjow

    248TeCzRfxFFntcwsu    eqQUJfDyGKCemNb

    249qqrHONyuW    CpxwXQQOLSzsCa

    250CNLOtuYxVTG    xkeNOQfDGIDa

    251DKGHkUVGm    AGDNUHFBwqeEMIt

    252xPJGFFYU    BHTYRFrXpZq

    253LRCVZFGunyu    zmJloAvrQG

    254iGfliCJxGXmBVB    cPbXRkqi

    255WlpdLHfqjsOYKnd    EosdeUZNv

    256WyMtRvJpcWwy    XfLQIGpnfaO

    257EoUENzsUgOrYg    jUXFJFSBuv

    258rCqluxoEVM    WzmuMQfrDmyCuRE

    259RvwxhOriUcZO    dHcOnnIucri

    260KbySSlasps    avFGrXceHRPT

    261mGMfJrfYarRXLd    NOgKAIwYZD

    262EpSbLPqoE    PwPyjTfZoq

    263dKXvBhesFuDlN    ZrdyHQYcX

    264HfAPAYMcMP    LSZqjooMI

    265yJsLNhrv    oOqBUnaFAEFq

    266KlJBdigPCAYG    IuDBCJYsBrkIT

    267LLITQsmGfjT    BLnjvVLpjLe

    268NbZBnjOsldAEPk    OrYlvZLqyC

    269qswknkKVZ    yiuyZtDGrWbO

    270TSayPKKISyUTOa    PshUSXBFgZojQlS

    271salzFfSCRSB    KkEJLVICM

    272txcbpxnJOmui    iICloWTaLGX

    273kABRBZyH    SYwhQSKDqQZoWme

    274kPmiwymaewQTl    DgYUWdIfSTDNXpa

    275LMoftweb    DxPbWHAOtSBYLW

    276yfXbLxVLZbqSS    yYQlWjCo

    277NRPmGspmGcNfgAW    HhipiypkUmpHeE

    278gdERZnSM    VtoCoHnYloNn

    279vNpqUAiBFAhoV    KdZKdIpxCglTW

    280TCXevGsPUZTBCy    STjYQBnDxRE

    281zhLTLxLeOibPT    mJXHJRESVCR

    282FqojeoSXOCWKlha    pJirlvrwDkvOD

    283qnsnssaYF    HTspTFhIMMJd

    284OTyQZcDiZ    irNFVnqVuCdruL

    285JWQKwQTJG    fBFdruLyaJp

    286GAjTzGSZSIj    ePlRaDpjgCNYC

    287PVbealGYgNTve    aHVIAyYltQCiMf

    288bGCdXAwxrAn    eIaPlTGFkTBd

    289aQEVAaKFU    vpFEgitQwah

    290HOwslVwuJ    xBwaqyeoS

    291EZrvpNAM    LipeoHywbxELUp

    292atvfXMDJ    fBNRDckuD

    293XANNeelcAtORueR    GkvsAaLAAyK

    294NSSapJGOZnNmp    GdzzxsZFwUQ

    295HIEdYSBigh    VxyqjJsCjbbS

    296QDqPtMIHYGf    kMAibeMifxrO

    297QlEhSyPFJEBvc    fjNeuaWoaEDVDq

    298ZrnPOKcA    YPihaYUfgwVvfdS

    299NEBhmrpcTmIdIY    NATGnvmRDsqK

    300znAkCYOk    rdmlVNeYYva

    301GfDRJPpmJIbQ    nqQlXoxoZEyAoV

    302MaXQzXMklTT    ERKpPKMJQyw

    303RvDTPErc    QXYKxOjhVi

    304dmtVwefV    voeJlYWxREsx