tag:blogger.com,1999:blog-8643857899806162280.post1077862767719975226..comments2024-02-25T06:15:55.318-03:00Comments on Bug squash: Indexing millions of documents with Solr and SolrNetMauricio Schefferhttp://www.blogger.com/profile/15247972578064164206noreply@blogger.comBlogger15125tag:blogger.com,1999:blog-8643857899806162280.post-42770307614864643472014-02-16T15:52:33.806-03:002014-02-16T15:52:33.806-03:00@Kanja: yes, but the important thing is to fetch t...@Kanja: yes, but the important thing is to fetch the records from the database <i>lazily</i>Mauricio Schefferhttps://www.blogger.com/profile/15247972578064164206noreply@blogger.comtag:blogger.com,1999:blog-8643857899806162280.post-78244880216093367672014-02-16T11:14:53.597-03:002014-02-16T11:14:53.597-03:00Thanks for the post Mauricio.
For this line, how ...Thanks for the post Mauricio. <br />For this line, how would I access the docs directly from the database.<br /><br />Parallel.ForEach(Partition(docs, 1000), group => solr.Add(group));<br /><br />Do I get the records from the database and store it in docs first before executing the Parallel comman above?Kanjanoreply@blogger.comtag:blogger.com,1999:blog-8643857899806162280.post-33962870659029800002013-11-20T09:58:00.652-03:002013-11-20T09:58:00.652-03:00Hi, thanks for your excellent article. We have bee...Hi, thanks for your excellent article. We have been looking for something that could improve indexing speed. I hope this helps. We'll try this as you advised. Thanks again.<br /><br />Krunal<br />nopAccelerate.comnopAcceleratehttp://www.nopaccelerate.com/noreply@blogger.comtag:blogger.com,1999:blog-8643857899806162280.post-79675930942933287792013-10-30T22:38:17.966-03:002013-10-30T22:38:17.966-03:00laszlo: Post all questions about SolrNet to the ma...laszlo: Post all questions about SolrNet to the mailing list https://groups.google.com/forum/#!forum/solrnetMauricio Schefferhttps://www.blogger.com/profile/15247972578064164206noreply@blogger.comtag:blogger.com,1999:blog-8643857899806162280.post-6144812657195266332013-10-29T15:01:42.866-03:002013-10-29T15:01:42.866-03:00Hi,
Could you advise me how to add xml formatted d...Hi,<br />Could you advise me how to add xml formatted data through solrnet.<br />To replace solr.Add(docs);<br />Thanks,<br /> laszloAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-8643857899806162280.post-13960200210294764482013-10-29T15:00:44.963-03:002013-10-29T15:00:44.963-03:00Hi,
Could you advise me how to add xml formatted d...Hi,<br />Could you advise me how to add xml formatted data through solrnet.<br />To replace solr.Add(docs);<br />Thanks,<br /> laszloAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-8643857899806162280.post-41119750138705199322013-09-10T11:07:54.524-03:002013-09-10T11:07:54.524-03:00@Satish Chandra Mishra : please ask all questions ...@Satish Chandra Mishra : please ask all questions about Solr in the Solr mailing list: http://lucene.apache.org/solr/discussion.htmlMauricio Schefferhttps://www.blogger.com/profile/15247972578064164206noreply@blogger.comtag:blogger.com,1999:blog-8643857899806162280.post-58323595507068611512013-09-10T11:01:28.541-03:002013-09-10T11:01:28.541-03:00I am trying to do the following...
1. On the face...I am trying to do the following...<br /><br />1. On the facet field of column A, I want to display the distinct count of column B matching the records with the value of column <br />A against facet field names. Simple facet field gives me the record count from the document instead of facet field A.<br />2. On the facet pivot table of columns A and C, I want to display the distinct counts of column B with matching values of facet pivot columns.<br /><br />To do so, I am trying to create multiple entities with detailed and distinct field details under the same entities under the single document<br /><br /><br />entity 1 -- detailed records including all fields.<br />entity 2 -- Disctint records for Column A and column B<br /><br /><br />when I am trying to load these entities, it overwrites the first loaded entity.<br /><br />Alternatively I am trying to create multiple entities under different documents but it doesn't the data either. It loads 0 records with no errors.<br /><br />I am new to Solr, Please advise, what should be my approach and steps.Anonymoushttps://www.blogger.com/profile/11517110769725689341noreply@blogger.comtag:blogger.com,1999:blog-8643857899806162280.post-64453079763920096542010-08-21T10:16:04.529-03:002010-08-21T10:16:04.529-03:00@Terrance: interesting, but if you shift the main ...@Terrance: interesting, but if you shift the main logic to a RESTful service you get basically what I describe in my article, an ETL process with DIH only acting as a light adapter. Anyway I would never expose this as a RESTful service by default unless I had a very good reason, this should remain internal to the application.Mauricio Schefferhttps://www.blogger.com/profile/15247972578064164206noreply@blogger.comtag:blogger.com,1999:blog-8643857899806162280.post-50898586021100893692010-08-21T02:25:59.371-03:002010-08-21T02:25:59.371-03:00I have been doing professional SOLR integration at...I have been doing professional SOLR integration at a large financial company and I can say for sure the DIH is perfectly fine when you take the following approach.<br /><br />What you are doing is pushing data into SOLR via ETL.<br /><br />Assuming you are using an RDMS might as well just use DIH over a database and be done with it.<br /><br />If you do have messaging to do, I imagine you do it everywhere, including the UI and other various places. You may even have a service layer that handles this.<br /><br />Point SOLRs DIH at a RESTful service using the same amount of code you would of wrote using the bulk operation you described. This time uou get a RESTful service out of it, and you get to use DIH out of it as well.Unknownhttps://www.blogger.com/profile/16143298805763839455noreply@blogger.comtag:blogger.com,1999:blog-8643857899806162280.post-27401486165423959282010-03-19T13:15:41.909-03:002010-03-19T13:15:41.909-03:00Nice grouped use of task partitioning and the para...Nice grouped use of task partitioning and the parallel task library. <br /><br />Just wanted to say thanks for the SolrNet info and library. I basically implemented my own, less awesome, version of Solr for a database indexing project.KevMhttps://www.blogger.com/profile/17217764002899797932noreply@blogger.comtag:blogger.com,1999:blog-8643857899806162280.post-54043084179643040742010-02-21T15:17:40.102-03:002010-02-21T15:17:40.102-03:00Great post.. This should help me with my solr re-i...Great post.. This should help me with my solr re-indexer that i've nearly finished writing.David Crafthttps://www.blogger.com/profile/05571714270796700086noreply@blogger.comtag:blogger.com,1999:blog-8643857899806162280.post-36116165211890762482010-02-18T04:08:30.075-03:002010-02-18T04:08:30.075-03:00This comment has been removed by a blog administrator.Unknownhttps://www.blogger.com/profile/13768224845029984324noreply@blogger.comtag:blogger.com,1999:blog-8643857899806162280.post-36931185385887173722010-02-10T12:24:43.156-03:002010-02-10T12:24:43.156-03:00I think your point about DIH can lead to "exe...I think your point about DIH can lead to "executable xml" is very key. DIH is great for integrating a database or other data source that doesn't require any massaging. But the more massaging you need, the better going with executable code seems to be! And I love you can use SolrNet and I can use rsolr!Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-8643857899806162280.post-67225149968869524242010-02-10T11:51:52.721-03:002010-02-10T11:51:52.721-03:00I'm building a large project with SolrNet and ...I'm building a large project with SolrNet and this should help a lot. Great post, thanks!Unknownhttps://www.blogger.com/profile/08805504301043969558noreply@blogger.com