Reviews, narrative and other documents for proposal

200301700: Integrated Status and Effectiveness Monitoring Program (ISEMP): The design and evaluation of monitoring tools for salmon populations and habitat in the Interior Columbia River Basin.

(View full proposal and assessments at

ISRP final recommendation: Meets Scientific criteria? Response requested

The project proponents have done a thorough job of addressing, point-by-point, the questions and concerns in the previous version of their Fast Track proposal. The response dealt primarily with the rationale for expanding the PIT tag array infrastructure in the Salmon, Grand Ronde, and Imnaha River subbasins. The ISRP appreciates the clarity of the response. When data were not yet available to answer some of the ISRP inquiries, the response identified information that was not available. With regard to the ISRP's question about why ISEMP was selected to act as the lead entity for the PIT tag array proposal, the response showed how ISEMP had already developed successful methods of obtaining, storing, retrieving, and analyzing data. Furthermore, they have developed good working relationships with other agencies/tribes in the area and were strongly supported by these organizations. With regard to the concern about whether the arrays were intended to compare methods (weirs, rotary screw traps, DIDSON dual-frequency sonar, etc.) or to inform salmon and steelhead managers, the response suggested that the emphasis would be on evaluating the accuracy and precision of PIT tag arrays relative to other methods currently being employed and also on assisting managers. They indicated that the proposed locations for installation of the arrays were selected primarily to fill data gaps on adult escapement for priority populations in the Snake Basin, a management function. The work augments testing of arrays that is ongoing in other Snake River subbasins by expanding the range of watershed and stream conditions where arrays are located, thus enabling testing under a greater variety of biological and physical conditions. The proposed work will complement studies, currently in the early stages, for determining the efficacy of arrays in detecting PIT tagged downstream migrants, which would likely be an improvement over conventional methods. The response also pointed out that the expanded network of arrays, if implemented, would provide at least one interrogation point for adults and juveniles of each life history type of each major population group for stream-type Chinook and steelhead. With regard to the question of whether all of the arrays could be installed prior to the 2010 field season, the response admitted that some of the proposed sites would have to be delayed until 2011, and that repairs to one of the existing arrays would also wait until next year. With regard to the ISRP's question of how sites were selected, the response adequately detailed the rationale for each of the proposed locations. While the proposed expansion of the number of arrays addresses a number of data gaps in the Salmon, Grand Ronde, and Imnaha River subbasins, the response also admitted that monitoring would likely continue to be inadequate for the Selway and Lochsa Rivers owing to their remote location and wilderness designations. On the other hand, the response suggested that if the PIT tag array technology proved to be the preferred method for monitoring adult and juvenile passage in large, turbulent rivers, arrays could be added to these two important "reference" watersheds (no supplementation and minimal habitat restoration). With regard to the question of whether the PIT tag arrays are capable of sufficient accuracy to calculate freshwater productivity, the response presented evidence - based on previous tagging and tracking research - that the approach appears to suffer from fewer problems than many of the other census methods currently in use. This conclusion applied to both adult and juvenile movement. With regard to the question of whether data collected to date could identify where limitations to freshwater productivity, the response argues that the PIT tag array technology can provide the greatest gains in understanding survival in large rivers that have been traditionally hard to sample. For example, they state that the expanded array will help answer the question of where subyearling stream-type Chinook rear when they emigrate from headwaters (i.e., in the mainstem Salmon River or in the reservoirs). Findings such as this would represent an important contribution to knowledge, although the PIT tag technology will not be able to identify the mechanisms of productivity constraints, but rather the arrays would help identify general locations in the subbasin where significant mortality occurs. In summary, the proponent has adequately addressed the ISRP's concerns and also indicated (although not in this response) that ISEMP will be available to present a summary of their findings to the ISRP/ISAB in late spring prior to the categorical RM&E solicitation. Because of the considerable importance of this project, we feel that ISEMP leaders should present periodic updates of key findings to the ISRP (e.g., every two years), including a summary of how the findings are being used in the management arena. We therefore look forward to ISEMP's presentation and anticipate we will be able to complete a scientific review of the ISEMP project prior to the categorical RME solicitation.

from Apr 2010 ISRP 2010-10 report

Narrative and other documents

2003-017-00 ISEMP PIT Tag Array Fast Track Review Response to ISRP Comments.docx
FastTrack_200301700_FY2010.doc (narrative)

ISRP preliminary recommendation: Meets scientific criteria? Response requested

The Columbia Habitat Monitoring Program (CHaMP) monitoring program described in this proposal is very comprehensive and ambitious. The fact that this effort builds on the success of the ISEMP project provides increased confidence that the program can succeed. As the program is implemented, participants should consider developing formal mechanisms for communicating results to mangers. The ISEMP project holds a great deal of promise for answering the questions: “What is the current status of fish habitat in the Columbia River Basin?” and “Are restoration actions currently being undertaken in the Columbia River Basin having the desired effects on both habitat condition and biological response?” We were pleased to see that the current proposal includes a number of new watersheds that will expand the geographical scope and completeness of ISEMP. We were also pleased that the strong emphasis in standardized data collection and spatially balanced and randomized sampling is intended to bring more consistency to monitoring efforts in the Columbia River Basin. In general, the proposal meets scientific criteria, with one qualification. The ISRP also offers additional suggestions for project proponents to consider. Qualification: The ISRP recommends that ISEMP organize a one-day workshop to discuss the CHaMP approach with the ISRP/ISAB and others. A draft of CHaMP should be circulated to the ISRP/ISAB before the workshop. Specific issues at the workshop should include how previously collected data can be or has been incorporated into CHaMP databases. It would also be useful to summarize how ISEMP priorities have evolved over the years, as well as a publication strategy. This qualification was discussed with the ISEMP project lead on September 13 and was determined to be agreeable. The workshop will be tentatively scheduled for late 2010 or early 2011. 1. Purpose, Significance to Regional Programs, Technical Background, and Objectives In the Columbia River Basin, there are a wide variety of RM&E projects that often differ in sampling design, methods, metrics, and quality and extent of data analysis, and often do not lead to definitive evaluations of habitat restoration effectiveness. ISEMP is a far reaching project that is based on the reasonable proposition that a standardized set of protocols, procedures, and data collection and analytical methods that can be adopted basinwide is needed to improve data collection precision and accuracy, provide comparability of results within and between subbasins, and so improve the capability of determining the effectiveness of habitat restoration projects. This proposition has been tested in a limited number of watersheds. In this proposal ISEMP is expanding its scope to include several additional watersheds to further evaluate its protocols for study design, data collection, analytical methods, and information dissemination. This effort is worthwhile in that improvements in habitat effectiveness evaluation are badly needed within the Basin. The technical background is very complete in the proposal. This project is clearly significant to regional restoration programs. A coordinated program for the collection, compilation, and archiving of data on fish populations and habitat condition has been identified in numerous ISRP and ISAB documents as a critical need for the Columbia Basin. Explanation of the significance of CHaMP to regional programs was very thorough. One question had to do with data archiving and database sharing. What will the interface between ISEMP databases and BPA’s Taurus project tracking system be? Will CHaMP data be linked to Taurus in such a way that interested parties can access habitat or population status and trend data (e.g., the CHaMP metadata library) directly, or will these databases be housed separately by ISEMP outside Taurus? It was gratifying to see that CHaMP will be testing novel remote sensing techniques for assessing habitat condition over large areas, e.g., using green LiDAR or multispectral sensing. This project has also demonstrated leadership in tracking population and life history attributes using PIT-tag arrays and other landscape-based methods. 2. History: Accomplishments, Results, and Adaptive Management The links provided to reports produced through ISEMP, in the aggregate, provide a very complete picture of the results generated by this project and reflects well on the prospects for success of the CHaMP program. The manner in which the CHaMP data will be used adaptively to modify future monitoring efforts is clearly described, and some examples are provided. However, the link with managers and policy decision makers in the basin is less clearly described. The proposal indicates that a process will be established specifically to utilize the data generated through CHaMP to produce new analysis tools, which will be used to generate the type of information required to determine future direction of restoration efforts and to support fisheries management decisions. But there is another step required to make this process maximally effective; a formal process for communicating the output from the data and analytical tools to non-technical audiences. ISEMP has used periodic newsletters as one mechanism for addressing this function. This approach also would be a reasonable option to consider for CHaMP. But the CHaMP project leaders should devote some effort to developing a consistent process for broadly disseminating program results. The ISEMP project has expanded in scope perhaps more than any other habitat restoration-related project funded by BPA since its inception. There are now, according to the proposal, ISEMP studies taking place in 26 watersheds in the Columbia River Basin, all of which contain anadromous salmonids. That organizers have succeeded in growing this project in such an impressive fashion reflects well on the willingness of a wide variety of stakeholders (federal and state agencies, tribes, local conservation districts) to enter into cooperative arrangements with the ISEMP project to address large scale restoration status and effectiveness questions. ISEMP has grown to such an extent that many of its component parts could be treated as separate projects. It was interesting to see how the proposal described past results. There were abundant maps and lists of activities taking place in ISEMP watersheds, but there were relatively few graphs or tables showing how target species have responded to habitat restoration. We were hoping for a little more in the way of biological response findings, since some restoration locations have now been monitored by ISEMP for seven years. The proposal suggests that the results of habitat restoration often require extended monitoring periods (i.e., often decades) in order for their effects to be assessed. We concur, but including a few highlights of some of the most informative results to date would have made the proposal more interesting. The proposal does an excellent job of describing the formation and evolution of the CHaMP effort, which is in effect an important type of adaptive management, i.e., the development of standardized habitat survey protocols in order to facilitate data analyses and inter-watershed comparisons. Overall, the description of other activities was thorough and informative. One adaptive management question is: have any restoration actions changed as a result of ISEMP findings? In particular, we are interested in knowing if anything is being done differently because evidence is starting to suggest that current approaches are not working as anticipated. Perhaps, as the proposal points out, it is premature to make judgments but if there are any good examples of restoration practitioners learning from past mistakes, they would be worth knowing. 3. Project Relationships, Emerging Limiting Factors, and Tailored Questions for Type of Work (Hatchery, RME, Tagging) Generally, the relationships of this program to other habitat and fish monitoring projects in the Columbia Basin are well described in the proposal. The one relationship that could have benefited from a more thorough discussion was the association between CHaMP and PNAMP. Several of the deliverables in the proposal will be co-developed with PNAMP. But the role of each organization in producing these deliverables was not clearly described. A paragraph in the introduction that outlines this relationship and some indication under the shared deliverables of roles and responsibilities would have helped to clarify the division of labor. The monitoring work includes tagging, other types of fish population assessment, experimental habitat restoration, long-term habitat trend monitoring, habitat protocol standardization, food web studies, and management of very large datasets. The project does not emphasize identifying limiting factors; rather, it is aimed more toward evaluating the responses of aquatic habitat and fish populations to restoration actions at large spatial scales. Overall, the proposal does a good job of relating ISEMP-sponsored monitoring to other restoration and monitoring projects. The restoration questions being addressed in each of the watersheds are appropriate to the issues believed to be limiting to salmon production. The new CHaMP rotating panel (GRTS) monitoring design appears to hold considerable promise in characterizing habitat status and trends. The project is consistent with the call for expanded RM&E in many subbasin plans and regional programs. This work is relevant to most RM&E efforts basinwide and provides a means by which RM&E programs in diverse subbasins can be unified under a common set of protocols and procedures. The ISRP suggests that pollutants not be overlooked as potentially limiting factors in certain locations. 4. Deliverables, Work Elements, Metrics, and Methods Deliverables, work elements, metrics and methods are adequately described. Some of the details specific to different work elements could be explained more completely (e.g., in what types of habitat will benthic macroinvertebrates be sampled, and why?), but given the very broad scope of the project and the need to cover all the work elements this is understandable. Journal publications are listed in many of the deliverables for different objectives, but to date there have not been very many papers published from the ISEMP work. Hopefully this will change in the near future. What was the rationale for including 25 sample sites in each basin for habitat status and trend monitoring? Did the ISEMP data suggest that this number of sites would provide adequate statistical power? Some support for this number of sites should be provided. It would seem that the number of sample sites required to adequately represent the range of channel types within a basin would vary based on watershed size, variation in topography, geology, land use and other factors. If this is the case, a variable sample size might be more appropriate. The assessment of stable isotope analysis to characterize trophic aspects of habitat condition in the John Day River is a novel element of this habitat monitoring program and could lead to the development of a very valuable tool. Limiting factors have almost exclusively been restricted to physical habitat or water quality attributes, largely because there was no efficient method for assessing food web conditions. One suggestion about the proposed methodology for this work element: the terrestrial invertebrates should not be lumped together for stable isotope analysis. As with the aquatic invertebrates, the terrestrial insects should be grouped on the basis of functional group (detritivores, herbivores, or predators). The deliverable that addresses status and trend monitoring in the Wenatchee actually discusses this activity in the Entiat. In fact, the information for the Entiat is repeated in the second deliverable, which actually does address the Entiat. This discrepancy seems like an inadvertent cut-and-paste error, but the Wenatchee information should be added to the first deliverable. ISEMP suggests, where appropriate, an “experimental approach” where habitat restoration actions in “treatment” streams are compared to reference “untreated” streams. Given the physiographic and biological variability within reaches or tributaries within a subbasin, let alone differences between subbasins, selection of appropriate references and treatments could prove challenging. It probably would be helpful if ISEMP provided guidelines and/or assistance to subbasin investigators for selecting both reference and treatment sites. The same could be said for data analyses. With the large amount of data that will be collected, investigators may need some assistance in data analyses. It is our understanding that ISEMP is planning to provide analytical assistance where needed.

from Feb 2010 ISRP 2010-7 report