Solr: Clone Solr Request Instance


In some cases, we may have to clone current SolrQueryRequest instances: we may want to send client solr request to multiple cores etc.

Copy SolrParms is easy. But how can we copy ContentStream? This code will not work: newReq.setContentStreams(oldReq.getContentStreams());.

As most types of streams can only be read once: for example HttpRequestContentStream: when we send data in a post request to Solr, the HttpRequestContentStream would contain the data.
If we clone current SolrQueryRequest objects to multiple instances, then only one instance can read the data, others will throw exception: stream close related exception.

To fix this, the key is to copy content from ContentStream to a StringStream first, a StringStream can be read multiple times.

The code looks like below:
public static SolrQueryRequestBase cloneSolrRequest(SolrQueryRequest oldReq)
  throws IOException {
 SolrQueryRequestBase newReq = new LocalSolrQueryRequest(oldReq.getCore(),
  oldReq.getParams());
 Iterable<ContentStream> iterable = oldReq.getContentStreams();
 if (iterable != null) {
   Iterator<ContentStream> it = iterable.iterator();
   List<ContentStream> newStreams = new ArrayList<ContentStream>();
   while (it.hasNext()) {
  ContentStream oldStream = it.next();
  if (oldStream instanceof StringStream) {
    newStreams.add(oldStream);
  } else {
    InputStream is = oldStream.getStream();
    String streamContent = IOUtils.toString(is);
    StringStream sStr = new StringStream(streamContent);
    sStr.setContentType(oldStream.getContentType());
    sStr.setName(oldStream.getName());
    sStr.setSize(oldStream.getSize());
    sStr.setSourceInfo(oldStream.getSourceInfo());
    newStreams.add(sStr);
  }
   }
   newReq.setContentStreams(newStreams);
 }
 return newReq;
}
To copy this SolrQueryRequest to multiple cores, we first clone old SolrQueryRequest to a StringStream based SolrQueryRequest, then copy SolrParams and ContentStream(StringStream) directly.
SolrQueryRequest convertedReq = cloneSolrRequest(req);
  private SolrQueryRequestBase copyReqToCore(SolrQueryRequest oldStringStreamReq,
      SolrCore core) throws IOException {
    SolrQueryRequestBase newReq = new LocalSolrQueryRequest(core,
        oldStringStreamReq.getParams());
    newReq.setContentStreams(oldStringStreamReq.getContentStreams());
    return newReq;
  }

Labels

adsense (5) Algorithm (69) Algorithm Series (35) Android (7) ANT (6) bat (8) Big Data (7) Blogger (14) Bugs (6) Cache (5) Chrome (19) Code Example (29) Code Quality (7) Coding Skills (5) Database (7) Debug (16) Design (5) Dev Tips (63) Eclipse (32) Git (5) Google (33) Guava (7) How to (9) Http Client (8) IDE (7) Interview (88) J2EE (13) J2SE (49) Java (186) JavaScript (27) JSON (7) Learning code (9) Lesson Learned (6) Linux (26) Lucene-Solr (112) Mac (10) Maven (8) Network (9) Nutch2 (18) Performance (9) PowerShell (11) Problem Solving (11) Programmer Skills (6) regex (5) Scala (6) Security (9) Soft Skills (38) Spring (22) System Design (11) Testing (7) Text Mining (14) Tips (17) Tools (24) Troubleshooting (29) UIMA (9) Web Development (19) Windows (21) xml (5)