Date   

Re: Welcome to EdgeX-DB-Persist@lists.edgexfoundry.org

qq
 

hello


15599633@...

 
Date: 2018-09-18 20:33
Subject: Welcome to EdgeX-DB-Persist@...

Hello,

Welcome to the edgex-db-persist@... group at EdgeX Foundry. Please take a moment to review this message.

To learn more about the edgex-db-persist@... group, please visit https://lists.edgexfoundry.org/g/edgex-db-persist

To start sending messages to members of this group, simply send email to edgex-db-persist@...

If you do not wish to belong to edgex-db-persist@..., you may unsubscribe by sending an email to edgex-db-persist+unsubscribe@...

To see and modify all of your groups, go to https://lists.edgexfoundry.org


Regards,

The edgex-db-persist@... Moderator


Reminder - meeting 9/4 @ 8am CDT

James.White2@...
 

The database persistence project group will meet at 8am CDT on 9/4.  Please find the connection information and agenda here:  https://wiki.edgexfoundry.org/display/FA/Data+Persistence+Project+Group.  It is likely that this meeting will close out the work of the database persistence project group for a time until work on database abstraction and 2nd database implementation is provided.

 


Database persistence project group recommendation

James.White2@...
 

All,

 

In this morning’s Database Persistence project meeting, we started to review a comparison matrix of potential database options and started to explore performance evaluation testing of databases (either with our own or with a known and regarded benchmarck).

While the work (which could be significant) to explore database alternatives and performance test the alternatives against one another could continue, it seemed that at this time (targeting Edinburgh release), it might be more appropriate to focus on the service architecture as it relates to persistence and allow for easier swap of the database by 3rd parties while also providing at least one other alternative to start to give examples of how to implement EdgeX services with alternate persistence mechanism.  Therefore, I proposed the following plan, with no objection from the project group, to take to the core working group for consideration at the next meeting:

 

oFor Edinburgh release, the core dev team works to implement a layer/abstraction system for databases.  Allowing any database to be used under the covers more easily.  Getting rid of any MongoDB specific ID or BSON references and isolating MongoDB specific to replaceable segments of the code base.

oFor Edinburgh release, we implement 2 options for the EdgeX reference implementation:  a MongoDB and a Redis set of core services.  this based on the fact that Redis aligns with many of the requirements (and is not an embedded database), shows good performance in benchmark tests and has Redis members participating as part of the project currently.

oFor Edinburgh release, we well document how to replace the database for 3rd party replacements.

oFor Edinburgh release, we have a certification process for core services – insuring any new core service adheres to the API sets and work appropriately in an EdgeX environment

oFor Edinburgh release, we have a marketplace for core services (something already underway via many areas of the project)

oFor Edinburg release, we have a performance test harness and set of benchmarks for core services that provide statistics on memory usage, CPU usage, speed of queries, writes, etc. to give the community a sense indications that may help in choosing an implementation that aligns best with their use case.

oFor the Fuji release, we make a determination of which of the reference implementations we want to keep long term (may be one or both or even take one out of the marketplace if it is shows itself to be better)

 

If you have opinions or suggestions about this plan, please reply to this thread and/or participate in the upcoming core working group meeting.

 

Thanks

 

Jim White

Distinguished Engineer, IoT Platform Development Team Lead

EdgeX Foundry Technical Steering Committee Vice Chairman

Dell Technologies | IoT Solutions Division

Office +1 512-723-6139, mobile/text +1 612-916-6693

james_white2@...

 


Database persistence project group to meet tomorrow 8/28 @ 8am

James.White2@...
 

Reminder that the Database Persistence Project Group meets tomorrow at 8am CDT.  Find connection details and agenda here:  https://wiki.edgexfoundry.org/display/FA/Data+Persistence+Project+Group.

 

I was able to do some work this past week on both a comparison matrix and performance test plan.  Hope we can discuss both and potential next steps in this project.

 

Jim White

Distinguished Engineer, IoT Platform Development Team Lead

EdgeX Foundry Technical Steering Committee Vice Chairman

Dell Technologies | IoT Solutions Division

Office +1 512-723-6139, mobile/text +1 612-916-6693

james_white2@...

 


documents for review

James.White2@...
 

Members of the Database Persistence project group,

As promised at our last meeting, I have a couple of documents for review that will be part of our discussion during the upcoming meeting next week.  Please find them here:  https://wiki.edgexfoundry.org/display/FA/Data+Persistence+Project+Group#DataPersistenceProjectGroup-Documents

 

1.       Comparison matrix

2.       Performance evaluation planning

 

After doing a lot of research in this area this week, I have some ideas about how we can move forward.  Look forward to the discussion next week.

Jim

 

 


DB Persistence project meeting tomorrow 8/21 at 8am CDT

James.White2@...
 

EdgeX’ers – reminder that the Data Persistence project group is meeting tomorrow (8/21) at 8am CDT.  Find connection  information and agenda here:  https://wiki.edgexfoundry.org/display/FA/Data+Persistence+Project+Group

 

Tomorrow’s topics include wrapping up requirements and discussing how to evaluate.  Look forward to your being on the call.

 

Jim White

Distinguished Engineer, IoT Platform Development Team Lead

EdgeX Foundry Technical Steering Committee Vice Chairman

Dell Technologies | IoT Solutions Division

Office +1 512-723-6139, mobile/text +1 612-916-6693

james_white2@...

 


Re: [Edgex-tsc-core] Data Persistence meeting and new mailing list

James.White2@...
 

Hi Srikanth,

Points all worthy of consideration and Eric from my team has suggested that at least adding one SQL database into the evaluation may make sense as others with SQL background could claim foul if we haven’t properly evaluated all the options.  With only so much bandwidth and resources, we will rely on the community to help with this as much as possible.  If we can find ardent SQL supporters like yourself that can help with evaluation and demonstrate the good, bad and indifferent parts of all the options, it will allow us to make the appropriate choices.  I have a sense, based on prior work, that while the JDBC/ODBC models and abstraction do allow SQL databases to be more easily swapped, the more structured data schemas make it harder to adapt to the wide range of IoT data types.  But I am always open to the exploration.  Next week’s discussions about how we start to evaluate and move forward with the exploration will be critical and getting volunteers to do the work will be even more critical.

Jim

 

From: Srikanth [mailto:kanth@...]
Sent: Thursday, August 16, 2018 12:39 AM
To: EdgeX-DB-Persist@...; White2, James
Cc: edgex-tsc-core@...; EdgeX-Devel@...
Subject: Re: [Edgex-tsc-core] Data Persistence meeting and new mailing list

 

Hello Jim, all:

 

Thanks for sharing the great notes. 

 

I was thinking that SQL maybe more than sufficient for the edge use-case.  It is not clear if there are benefits to noSQL options mentioned especially in single node type configurations.  Do we anticipate edge needs to really scale so large like they may in a cloud/centralized system that may need to serve the "whole world"-type scale?

 

Regardless, priority wise, does SQL seem the way to go.  Also, IMO, only SQL meets the requirement to easily swap datastores in a compatible way.  And, noSQL generally moves lot more complexity to the application side - which is totally avoidable for edge centric need.

 

-Srikanth

 

On Tuesday, August 14, 2018, 7:26:54 AM PDT, James.White2@... <James.White2@...> wrote:

 

 

For those that participated in today’s call on EdgeX data persistence – thank you.  Good call.  The minutes and recording from the meeting can be found here:  https://wiki.edgexfoundry.org/display/FA/Data+Persistence+Project+Group

 

As you can see by this email, we have a new EdgeX mailing list.  I will be using this mailing list going forward (as opposed to the Core mailing list).  Please feel free to use this new mailing list to provide any feedback, observations or ask questions regarding data persistence.

 

Regards,

Jim White

Distinguished Engineer, IoT Platform Development Team Lead

EdgeX Foundry Technical Steering Committee Vice Chairman

Dell Technologies | IoT Solutions Division

Office +1 512-723-6139, mobile/text +1 612-916-6693

james_white2@...

 


Data Persistence meeting and new mailing list

James.White2@...
 

For those that participated in today’s call on EdgeX data persistence – thank you.  Good call.  The minutes and recording from the meeting can be found here:  https://wiki.edgexfoundry.org/display/FA/Data+Persistence+Project+Group

 

As you can see by this email, we have a new EdgeX mailing list.  I will be using this mailing list going forward (as opposed to the Core mailing list).  Please feel free to use this new mailing list to provide any feedback, observations or ask questions regarding data persistence.

 

Regards,

Jim White

Distinguished Engineer, IoT Platform Development Team Lead

EdgeX Foundry Technical Steering Committee Vice Chairman

Dell Technologies | IoT Solutions Division

Office +1 512-723-6139, mobile/text +1 612-916-6693

james_white2@...