Thursday, March 10, 2016
Load balancers are the preferred solution for providing scalability and fail-over for Dgraph queries
An Endeca-based application based on the availability of the MDEX(Dgraph) Engine to serve user requests. If that Dgraph is unavailable, then the Endeca part of the application will not be able to respond to queries. The Dgraph might be unavailable for multiple reasons, including hardware failure, an in-process update of the Dgraph indexes, or very high load.
For high traffic sites, it may be necessary to have more than one Dgraph to serve traffic. For these reasons, it provides high availability and performance.
Routing
Generally, the best routing algorithm for balancing traffic between Dgraphs is to use the "least connections" model. This takes into account the variability in the time it takes an MDEX Engine to respond to a query, and thus balances requests in a slightly smarter way.
Health Check
Use below query to do dgraph health check:-
http://[host]:[port]/admin?op=ping - This is very light weight query.
The load balancer should be configured to check for a response within an appropriate duration.
Query Failover
Upon failure of a health status check, the load balancer should be able to mark the dgraph port as unavailable and remove it from sending the query to dgraph engine.
But still there are chances that the load balancer will send a request to an offline Dgraph even health check is 5 seconds.
Actively Managing the Load Balancing Services through Scripting
Many load balancers support a scripting interface to let us programmatically disable and enable ports. This allows the option of proactively disabling a Dgraph port in the load balancer before shutting the Dgraph down for a baseline update refresh.
<dgraph id="Dgraph1" host-id="MDEXHost" port="15000" pre-shutdown-script="DgraphPreShutdownScript" post-startup-script="DgraphPostStartupScript">
<properties>
<property name="restartGroup" value="A" />
</properties>
<log-dir>./logs/dgraphs/Dgraph1</log-dir>
<input-dir>./data/dgraphs/Dgraph1/dgraph_input</input-dir>
<update-dir>./data/dgraphs/Dgraph1/dgraph_input/updates</update-dir>
</dgraph>
Above functionality is typically used to implement calls to a load balancer, adding or removing a Dgraph from the cluster as it is updated.
Click here to view Endeca Query Performance issues and Fixes.
From Author
Was this post provide the solution that you are looking for? what do you think, is there anything else that can be applied as load balancer practices for Endeca Dgraph? Provide your comments
An Endeca-based application based on the availability of the MDEX(Dgraph) Engine to serve user requests. If that Dgraph is unavailable, then the Endeca part of the application will not be able to respond to queries. The Dgraph might be unavailable for multiple reasons, including hardware failure, an in-process update of the Dgraph indexes, or very high load.
For high traffic sites, it may be necessary to have more than one Dgraph to serve traffic. For these reasons, it provides high availability and performance.
Routing
Generally, the best routing algorithm for balancing traffic between Dgraphs is to use the "least connections" model. This takes into account the variability in the time it takes an MDEX Engine to respond to a query, and thus balances requests in a slightly smarter way.
Health Check
Use below query to do dgraph health check:-
http://[host]:[port]/admin?op=ping - This is very light weight query.
The load balancer should be configured to check for a response within an appropriate duration.
Query Failover
Upon failure of a health status check, the load balancer should be able to mark the dgraph port as unavailable and remove it from sending the query to dgraph engine.
But still there are chances that the load balancer will send a request to an offline Dgraph even health check is 5 seconds.
Actively Managing the Load Balancing Services through Scripting
Many load balancers support a scripting interface to let us programmatically disable and enable ports. This allows the option of proactively disabling a Dgraph port in the load balancer before shutting the Dgraph down for a baseline update refresh.
<dgraph id="Dgraph1" host-id="MDEXHost" port="15000" pre-shutdown-script="DgraphPreShutdownScript" post-startup-script="DgraphPostStartupScript">
<properties>
<property name="restartGroup" value="A" />
</properties>
<log-dir>./logs/dgraphs/Dgraph1</log-dir>
<input-dir>./data/dgraphs/Dgraph1/dgraph_input</input-dir>
<update-dir>./data/dgraphs/Dgraph1/dgraph_input/updates</update-dir>
</dgraph>
Above functionality is typically used to implement calls to a load balancer, adding or removing a Dgraph from the cluster as it is updated.
Click here to view Endeca Query Performance issues and Fixes.
From Author
Was this post provide the solution that you are looking for? what do you think, is there anything else that can be applied as load balancer practices for Endeca Dgraph? Provide your comments
No comments
Post a Comment
Note: Only a member of this blog may post a comment.